A walk in the forest of worktrees

Lately, I’ve been working on migrating Doctrine ORM to PHP 8 syntax. To that end, I’ve been using Rector, an automated refactoring tool. It comes with a set of rules called LevelSetList::UP_TO_PHP_81 which makes sure you use the most modern syntax to do something as long as it is supported on PHP 8.1. UP_TO_PHP_81 is equivalent to UP_TO_PHP_80 + PHP-8.1-specific rules. UP_TO_PHP_80 is equivalent to UP_TO_PHP_74 + PHP-8.0-specific rules. UP_TO_PHP_74 is equivalent to… you get the idea, and maybe also marvel at how satisfying this looks. 🤓

I’m doing that work on the 3.0.x branch because we don’t currently plan to drop support for PHP 7.1 on the 2.* branches. While I’m at it, I’m taking this as an opportunity to add type declarations where not already present.
Adding type declarations is very often a breaking change, especially when working on methods that are not private, and classes that are not final,which explains why that was not already done everywhere on lower branches.

Thankfully, we have plenty of phpdoc comments that Rector can use to infer the correct type declarations to add. Here is how such changes might look:

- /**
-  * @param string $foo
-  *
-  * @return int
-  */
- public function doStuff($foo);
+ public function doStuff(string $foo): int;

doctrine/orm is a lot of code though, so I’m trying to work in bite-sized pull requests. First, because it would be awful to review all these changes at once, but also because the phpdoc we have can be imprecise, or plain wrong (we still have to work through our PHPStan and Psalm baselines). Having inaccurate phpdoc might be fine from PHPUnit‘s point of view, but having inaccurate type declarations isn’t, so I need to fix these by hand afterwards.

I like to repeat that we’ve never been closer to releasing doctrine/orm 3.0 than we are today, an information that you can share widely because it’s always true. While that’s a fact you can hardly deny, it is still good to backport any of the fixes and improvements to the 2.x series: it makes difference between
branches smaller, which in turn makes merging up from 2.x to 3.x easier, but also lets the users benefit from those fixes and improvements earlier.

Usually, what I do is pick a class or namespace, apply Rector on it, then review the changes. If I spot phpdoc that is wrong, I fix that on the patch branch (currently 2.13.x). If I spot phpdoc that is correct but a bit vague, I make it more precise on the minor branch (currently 2.14.x). After the PR is merged, I merge 2.13.x up into 2.14.x, and 2.14.x up in 3.0.x, and I try running Rector again, this time with correct phpdoc.

Contributing the right thing to the right branch

That migration is a good use case for the git subcommand I want to introduce to you today, because I need to change branches often. It’s already not unusal to have to do so when maintaining a library, but it’s exacerbated here. Sorry for the Tom Jones earworm.

In this particular case, here is the problem I am facing: imagine you have 10 fixes or improvements to backport from one branch to the others, and that you discover them progressively. How would you proceed? Would you stash changes, switch branches, run composer update for good measure, make your change, commit, then switch back every time? Or would you maybe try to remember several things you need to do, and try to do them all at once? Either solution sounds pretty bad.

The subcommand that can save you from this is git worktree.
It allows you to have ✨several✨ worktrees at once for a single repository.

Creating a throwaway worktree with branch 2.14.x can be done like so:

$ git worktree add /tmp/throwaway 2.14.x

The operation is instant, and in the case of doctrine/orm, there are only 2 steps to be ready to work on that new branch:

$ cd /tmp/throwaway
$ composer update

But I do not want to create throwaway worktrees… I find the idea of having permanent worktrees very appealing: they are like starting points for new branches I want to create. Each one has its own vendor directory, with the right dependencies.

Also, I prefer to have them neatly grouped in a single directory. I could have a normal repository, and then add worktrees inside, but then git would consider the worktrees themselves as new directories that need to be put under version control. To avoid having that main worktree, you can use a bare repository:

$ git clone --bare git@github.com:doctrine/orm.git doctrine-orm.git

You will end up with a directory called doctrine-orm.git, and the contents of that directory will be what you usually find in the .git directory If you use git log, you will see the history of the default branch, which the current HEAD points to (2.13.x in our case).

Doctrine uses a consistent branching model on all of its repositories:

  • 🐛 bugfixes go to the patch branch;
  • 💡 new features, deprecations, improvement go to the minor branch;
  • 💥 breaking changes go to the major branch.

At first, I named directories after branches, but when 2.12.x went unmaintained, I no longer had a use for the corresponding directory, and realized I should have one directory per branch type instead. Here is how to create that workforest 🌳🌳🌳:

mkdir ../doctrine-orm
git worktree add ../doctrine-orm/patch 2.13.x
git worktree add ../doctrine-orm/minor 2.14.x
git worktree add ../doctrine-orm/major 3.0.x

After that, you should end up with something like this

doctrine-orm
├── major # a full 3.0.x doctrine/orm is inside 💥
├── minor # a full 2.14.x doctrine/orm is inside 💡
└── patch # a full 2.13.x doctrine/orm is inside 🐛
doctrine-orm.git # Looks just like a regular .git directory
├── config
├── HEAD
├── hooks
├── objects
├── packed-refs
├── refs
└── worktrees # contains administrative files for your worktrees

Note that I could have created the worktrees directly inside doctrine-orm.git, but I don’t want to, I find that messy.

When inside doctrine-orm/*, git still knows where the repository is stored thanks to a .git file in each worktree. Yes, in this case it’s just a one-line file, with a pointer to a directory that holds administrative data for that worktree.

$ cat .git
gitdir: /path/to/doctrine-orm.git/worktrees/minor

When trying this at first, it broke custom git hooks I had. That’s because when using worktrees, Git will store the administrative data that is common to all three worktrees in the usual directory, but will put worktree-specific administrative data in another directory (here: /path/to/doctrine-orm.git/worktrees/minor). Making the distinction between the git directory specific to a worktree and the git directory shared by all worktrees helped me fix my hooks. It is possible to figure them out from inside a worktree:

$ git rev-parse --git-dir
/path/to/doctrine-orm.git/worktrees/major
$ git rev-parse --git-common-dir
/path/to/doctrine-orm.git

When directly inside doctrine-orm, nothing special happens, and you will get the usual error message when trying to issue a git command.

$ git status
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

When inside a worktree, you can know the status of all worktrees:

$ git worktree list
/path/to/dev/doctrine-orm.git    (bare)
/path/to/dev/doctrine-orm/major  e9f3a43f3 [php8-migration-persisters]
/path/to/dev/doctrine-orm/minor  cc9e456ed [10238--lockMode]
/path/to/dev/doctrine-orm/patch  28cb24b3c [psalm-5-fixes]

What does this unlock?

  1. Switching branches becomes as easy as switching directories. No need to commit or stash anything.
  2. I do not need to run composer update all the time when switching from one worktree to another.
  3. If I want to, I can even compare and edit the same file on two different branches at once. 🤯

I like that git has such hidden gems, like git worktree or git bisect, that are little known and rarely needed, but are still killer features when you do need them.

Community Driven Development

When talking about development I mostly mean personal and educational development. So this post is not about a new fancy way of writing software.

Why Community

I personally felt the power of a community on my way to the position I’m currently at. I started working with zero practical experience at small company in Germany. It was a good way to get into the business, but very soon I realized, that there is an other world out there. So I started experimenting with Symfony. At the same time I heard a conference talk by Lukas about the Symfony CMF. As content management was my daily business, I tried to dig into that Content Management Framework. I admit, I couldn’t understand how it worked. My first attempts on implementing it had been a disaster. But then I started committing to that ecosystem. It had not been my first contribution, but it was the first that ever got merged into something bigger. But that is not the point. While working on my own CMF component and committing to the main packages, I had an impressive learning experience.

When Working on such an project I learned a lot of the internal processes of Symfony. I really had helpful but demanding guys on every review. And indeed I would like to thank David and Wouter. With their help I made my biggest step in my personal development. Instead of declining a pull request, I got a lot of helping words. Those words contained more and better advice than a documentation can ever give.

So I became a core developer of the CMF. Currently I prepare a new major release with the same guys. We are planning to serve before Christmas. And again I learn. I currently don’t work on a Symfony application in my job. But with the work for the community I stay in touch.

Not your Story?

I believe, that not every open source project can bring on the same success, but most of the community driven projects do. Almost every bigger project has got chats, user groups or meetups and sometimes their own conferences. That can be an easy starting point. When using those projects in daily business, you should visit the community one way or the other. You will gain insights you will never get by simply reading documentation. Those visits combined with public speaking is the first contribution to such a project. There is no code or documentation involved yet, but it gives every internal contributor some self confidence. Realizing that one or more people use the code you wrote or contributed to is a really good feeling. Trust me. I stand in front of more than 500 Symfony developers and asked who knows or got in touch with the CMF. And almost the complete crowd raised their hand. I can’t really describe what that did to me. I is absolutely amazing.

Why not starting to commit?

Most often it starts with a bug or missing feature when using an open source project. Instead of ranting you should fork, fix and push. Maybe you realize why that bug had happened or the feature hasn’t been implemented yet. It is easy to yell for a specific behavior of an application or library, but the implementation of it can be more difficult than expected. Libraries used by lots of developers need a special kind of extensibility. Contributing to those libraries is the best way to understanding those concepts. And doing so, it also helps to get aware of the work, that is needed to get a feature on the road.. That said I have to add, that the code isn’t the only possible contribution.

The Documentation often needs some effort. So how often did you say “This isn’t documented well”? Fork, fix and push! Same answer here.

It is open source. It is implemented by a community of moms, dads and other people, who sacrifice their free time. There is no service hotline you can call and shout “I paid a lot of money for that, so it has to work the way I want.”. In my opinion there has to be a kind of compensation. The more you take, the more you should give.

Commit while your job

Yes! You should discuss in you company about a way of contributing to open source projects. It is not only about reinvesting. It is mostly about bringing company’s developers forward. As I described some lines above, I learned so much contributing to open source projects. Employees can too. Instead of spending a lot of money on workshops, get in touch with the community, contribute and learn. It is much cheaper. And what can be better marketing for a company selling experts in Project X, than having contributors to the same project on your team?

Conclusion

There are several ways of learning programming language specific skills, but contributing to and working on open source projects, you will also meet kind people and get a lot more information than just reading documentation.

Explore your local communities

Since moving to Munich in April I started attending various tech meetups. I had to wait for the next PHP meetup till the end of May (because the previous one was held late in March) but it was totally worth it – around 40 people attended, pizza and beer were tasty and the talks were about interesting topics. By chance the next meetup was, because of last minute speaker cancellation, a chance for me to have a lightning talk about HomeBrew PHP. I’ve attended other PHP-related user groups such as Magento (as a speaker) and Symfony. There is also Laravel group which I plan to check soon. Later in October I started my own UG about Phalcon framework

 

To sum it up: if you are new to the area just visit meetup.com website, chances are high there are established user groups for various interests including PHP. In case there are none, feel free to start one (and add it to the PHP.UserGroup as well!). Don’t feel a stranger, embrace and share the knowledge, meet new people and just have fun 🙂

 

PHP as a language and as a community had a great year and clearly has a bright future, lets all keep up the pace in the 2016! I wish all the readers a nice Holidays season

The PHP revolution in 2015

If you were a PHP developer, you can state that 2015 was an epic year! Many things occured this year that had small or big effects in your role as PHP developer.

20 Years of PHP

First of all 2015 is the year we can celebrate 20 years of PHP as Rasmus Lerdorf released his first version of PHP/FI on June 8, 1995. Back then PHP was nothing more than a bunch of C classes that allowed Rasmus to build dynamic web applications in an easier way than was possible with CGI driven Perl sites. Little did he know that his contribution would change the world.

Today PHP is responsible for more than 80% of all web sites on the Internet, maybe even a bit more if we were able to look behind firewalls and proxies. Businesses depend heavily on PHP for their main source of income (selling products or services) or have PHP become part of their business intelligence (ERP, CRM, CMS) or even beyond regular computer usage (e.g. license plate recognition, traffic monitoring or even control bridges).

I only got in touch with PHP in 2001, but I’ve seen the technology and the community behind it grow into an awesome ecosystem where there’s a general behaviour of sharing and doing good. I feel myself privileged to be a part of this wonderful community.

Major Version Milestones

Only last month we could already enjoy the release of Magento 2, the major e-commerce platform build on PHP, Drupal 8, the leading CMS platform build ontop of Symfony Framework, and Symfony 3, the leading PHP framework to build robust, dynamic web applications.

It’s not done yet! Members of PHP core have completed and tagged PHP 7.0.0 on December 1, the biggest milestone change since PHP 5 dating back in July 2004. If you want to know what’s new under the hood of this release, I can recommend getting the free e-book “Upgrading to PHP 7” by Davey Shafik.

The years ahead

With these new releases available, and the global trend to connect everything together over the internet (IoT), we will see more API’s being created to streamline all of this. With PHP 7 you get extra power and performance to compete with other technologies. 

Improved security tools to encrypt privacy data, hashing of passwords and secure communications between services and devices makes PHP a technology that will provide jobs and opportunities for decades to come. If you’re a student and like adventure, PHP might give you an exciting future!

Get on board and start building the future today!