PHP and the secrets of emergent architecture

Developers don’t like PHP

According to StackOverflows Developer Survey, 58,4% of all PHP developers would like to move to another Language (https://insights.stackoverflow.com/survey/2018/#technology-most-loved-dreaded-and-wanted-languages).

They show it when they write PHP code, too: There is no other language getting close to PHP when it comes to the usage of words like “crap”, “fuck”, “hate” or “shit”:
https://github.com/Dobiasd/programming-language-subreddits-and-their-choice-of-words

https://github.com/Dobiasd/programming-language-subreddits-and-their-choice-of-words

The reasons for this dislike are well documented in online rants, for example in https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/:

  • PHP is full of surprises: mysql_real_escape_string, E_ALL
  • PHP is inconsistent: strpos, str_rot13
  • PHP requires boilerplate: error-checking around C API calls, ===
  • PHP is flaky: ==, foreach ($foo as &$bar)

He is right: these APIs are neither predictable, nor consistent, nor concise. Current APIs in other programming languages tend to be a lot better nowadays.

Nevertheless, PHP is still used for 46% (https://webspotter.io/category/platforms) or 79% (https://w3techs.com/technologies/overview/programming_language/all) of all dynamic websites as the server side language of choice.
PHP or the spin-off Hack is the base technology for some of the largest internet sites:

  • Facebook
  • Yahoo
  • Wikipedia
  • WordPress
  • Slack

PHP is even, being a pure web language, on place number 4 of the GitHub language popularity index (https://octoverse.github.com/projects#languages)

What is going on here? Why do these companies still use PHP? Are they just stuck with a bad technology decision they made in the beginning before they got famous?

Cathedrals and Bazaars

1997, at the international Linux day in Wuerzburg, Eric S Raymond told the story about “The Cathedral and the Bazaar” (http://www.unterstein.net/su/docs/CathBaz.pdf) for the first time.
He describes to major strategies to develop software: the cathedral strategy and the bazaar strategy.

The Cathedral

ESR writes:

In the cathedral-builder view of programming, bugs and development problems are tricky, insidious, deep phenomena. It takes months of scrutiny by a dedicated few to develop confidence that you’ve winkled them all out. Thus the long release intervals, and the inevitable disappointment when long-awaited releases are not perfect

Cathedrals need architects, they need a good structure and a big design upfront. The design of the cathedral is important, there is no room for variation, for spontaneous ideas of the people building it.

http://media.openclipart.org/people/rones/500px-Cathedral-of-the-Ascension-color-by-Rones.png

The Bazaar

In the bazaar view, on the other hand, you assume that bugs are generally shallow phenomena – or, at least, that they turn shallow pretty quickly when exposed to a thousand eager co-developers pounding on every single new release. Accordingly you release often in order to get more corrections, and as a beneficial side effect you have less to lose if an occasional botch gets out the door.

There is no architect for a bazaar. Somebody provides space, sets up a number of rules and invites people to participate.
The bazaar is free. If you see that there is a certain product missing on the market, you can simply open up your own stall and wait for people who would buy it.

Linux is based on the idea of the bazaar. ESR even calls this model “the Linus law”.
In the first days of Linux, a lot of people did not believe that Linux would work out. Everybody can participate? Everybody is able to add a feature, drivers, modules, and functionality? This is going to end up in an inevitable mess. People will introduce bugs and unforeseen bad interactions since they don’t understand the whole software. Linux is going to collapse under the weight of uncoordinated efforts!
Bazaar

Today

Today we know that “Linus Law” worked out. Linux is not less stable, in contrary, it’s used for the majority of mission-critical systems. It’s the foundation for the major cloud providers today.
The bazaar model itself was a major success, too: Github is basically a technical implementation of it.
The late success of the most popular programming language today, javascript, is based on nodejs and npm – another bazaar.
PHP has been a bazaar right from the beginning. It was always easy to participate and provide extensions to the language. I have been working with several release managers and core developers of PHP. If you know what you are doing and willing to discuss your ideas with other developers it was always easy to play every role in the PHP world.

Agile Architecture

Cathedrals and Big Design Upfront

The original idea of software architecture was a cathedral approach: start with a proper, well designed and understood big design upfront. Start to implement after you defined how your architecture is going to look like.
If we don’t know how our software is going to look in the end, how should we define the architecture for it now?
Maybe somebody invents new technology that is going to be useful for our platform in 3 months? Maybe we discover that one of our base technologies does not live up to the promise it made?

We know that we need to be prepared for change when it comes to internet applications. Youtube started as a video dating platform, but the customers rather wanted to upload videos. Slack was an interactive game, but the chat appeared to be more useful. Twitter was a podcasting platform before iTunes came and they had to reinvent themselves. Paypal was a palm pilot app that nobody used, Pinterest was a mobile shopping platform with no demand. None of them would have survived if they would have kept their initial architecture.
But how do we create an architecture if we already know everything is going to change?
Plan

Emergent architecture

Agile development answered with emergent architecture (see https://submissions.agilealliance.org/system/attachments/attachments/000/000/2342/original/Emergent_Architecture_-_Just_Enough_Just_in_Time.pdf). We’ll figure out the right architecture on our way to the final software. KISS (keep it simple, stupid), DRY (Don’t Repeat Yourself), YAGNI (You Ain’t Gonna Need It) are parts of an emergent “just enough architecture” strategy.
In emergent architecture, it’s ok to change your mind and choose another architecture a few sprints later. It’s ok to try out new architectural options as “Spikes”.
But emergent architecture is not cowboy coding, where everybody implements whatever he wants – it’s an architecture that allows changing your mind.
It’s not possible to change your mind if your software isn’t flexible if it’s not modular. It’s not possible to change your mind if your software is entangled. It’s not possible to change your mind if you rely on a certain component, like a special database system.
The easier – and cheaper – it is to change your mind, the better the architecture can adapt to changed or newly discovered architectural requirements. Strategies like the SOLID design principles or microservices try to achieve exactly this. If it’s hard or expensive to change your mind it’s not emergent architecture, since it relies on learning, adapting and trying out new things.
If everything works out you end up with the architecture that is the best fit for your current requirements – and not, like in Big Design Upfront architecture – the best fit for the idea of the software you had before you even started developing it.

Bazaars are distributed emergent architectures (on steroids)

The bazaar model already implemented a lot of emergent architectural patterns, like the Unix principles, that have been a major influence on microservice design.
And it implemented emergent architecture in an even more powerful way – as something that was free, attractive, open and distributed. Why evaluate 2 spikes at the end of the sprint if you can have 10 people implementing their ideas of a good solution for free and trying them out for months? Why not adding other peoples brainpower to your ideas and implementation and have them suggest – or even provide – better implementations?
It’s useful to be able to choose one from multiple solutions for a certain problem. If a lot of people do this it’s even possible to figure out what solution is the best fit for most of the problems, simply by looking at the number of downloads or GitHub stars.
Bazaars are emergent architectures on steroids. They provide a multiple of implementations and empirical data, a multiple of learning and adaption.
Does this kind of architecture result in beautiful implementations? This just happens if beauty is the major driver to choose an implementation. Most of the time it is not, it’s simply the most usable, the most useful implementation. Whatever the benefits of using a certain module are, the one with the best overall fit is going to be the most popular.

PHP as an application of distributed emergent architecture

How do You create a platform that works well with emergent architecture?
Obviously, it needs to allow experiments, and these experiments should be fast and easy to implement.

That’s how PHP does it:

Existing open sourced functionality

There is already a mountain of existing PHP functionality out there. It is GitHubs fourth most popular language, and most of the repositories are free. There are 4-5 major solutions for every kind of problem, be it CMS, e-commerce or frameworks, with large infrastructures with hundreds of additional plugins and modules behind them.

PHPs builtin functions

PHP comes with thousands of builtin functions for the average setup. You need something? It’s probably already there. Is this good design? Certainly not! Will you be able to move fast? Yep.

Focus on functionality, not on parsing HTTP, restoring session variables, typing or memory management

When the PHP script starts all variables, request or session variables, are already there. By default, you don’t need to care about typing or memory management. There is no HTTP-event loop, everything is already done. With fewer things to care about you get faster and cheaper experiments.

Fast feedback for local development – the inner loop

Hot Module Replacement? The fast save-reload-check-cycle was a part of PHP right from the beginning. The default delay until a developer sees if his idea actually works out is 1 second.

Fast feedback for teams – the outer loop

PHP deployment is file-based and immediate. Update the file and you’ll see the changes with the next request. There are no compile steps, no build artifacts needed to see a change another developer created on your machine or in the test environment. A simple file transfer – most of the time as a part of the version control system – is enough.

Fast integration of existing libraries

If it can not be solved in PHP code, it’s easy to create a new PHP Extension. It’s even possible with limited C knowledge, and most of the times a few lines of code are enough to integrate a function.

Star Treks BORG as a role model

The PHP culture is not afraid of stealing ideas. If something looks interesting, it will be assimilated. If there is a nice idea in the Java world, in Golang, in rust, in ruby or python – we are going to steal it.

All these strategies support PHPs bazaar architecture.
They make PHP a marketplace of ideas. It’s not build to look beautiful as a cathedral. It’s not there of aesthetic purposes. But it is made to provide fast easy solutions for most of the requirements you have.
And if there is a solution missing for exactly your problem, most of the time it’s easy to create it.

So, why – and when – should i use PHP?

PHP is optimized for environments where you don’t know for sure what is going to happen next. Not by design – by emergent practice. By the survival of the fittest for the purpose.
PHP could not provide fast and cheap solutions and feedback loops for most of the problems if the focus would be beautiful design instead of, well, fast and cheap answers and feedback loops.

If you know exactly how your future will like – don’t use PHP. But if your future is unpredictable and risky – PHP will help you to figure out what works and what doesn’t in a very efficient way.