Sustainability in Software Engineering

Let’s start out with a clear statement: our entire IT infrastructure is built on Open Source. Many of the basic tools in IT, from small libraries to entire operating systems, are developed and maintained by people with a passion for software development, often in their free time. It started out that way because they enjoyed hacking on things to make them better. Some of them were able to make this their jobs and earn a more or less comfortable living. But the reality is that most people haven’t: some still work on those projects in their free time, and some have had to give up.

When I talked to Derick Rethans, the author and maintainer of Xdebug, during my last job interview I started thinking about our reliance on Open Source and the strain this puts on maintainers. Xdebug is an absolute must-have for every PHP developer. One of the first things I do when I set up a new project is to make sure I can debug using Xdebug. With this, I’m able to quickly debug an application and find the problem by slowing down the code and thinking along with what’s happening. I evaluate variables and statements and go up and down the call stack to see how I ended up in the current frame. If I were to sprinkle debug output throughout the code and go through the log files, this process would take significantly longer. Xdebug probably saved my various employers thousands of Euro over the past years by speeding up my development workflow. Have any of my companies ever given even a tiny percentage of that money to Derick, if only to show appreciation for his work? No. Have yours?

As much as I want companies to fund Open Source, I also understand their point of view: for most businesses, the goal is to make as much profit as possible. For most startups, the goal is to become profitable quickly before the money runs out. So if companies can save a buck by using a free solution and avoid license and other costs, they will do so. Just look at your typical tech stack: Linux is used as operating system, Apache or nginx as web servers. Install haproxy as a load balancer in front of it and PHP as scripting language behind it. Our stack needs a database, so let’s install MySQL or PostgreSQL. Caching might also be useful, so Memcache, Redis, or Varnish will be installed in some combination. To avoid having to re-invent the wheel, we’ll start out using Symfony and Doctrine, install some open-source bundles and write some custom code. Sound familiar? You can build an entire business without having to pay for a single license. You don’t even have to buy hardware if you’re comfortable running your application “in the cloud”. By using docker and kubernetes the system administrator becomes obsolete, and as an added bonus the company can slap a “devops” buzzword on its job offers.

Open Source is a Slippery Slope

While these things have no cost for a company, they’re not entirely free: someone else is paying for them. In the worst case, the developer of the tool is paying for it with his time. This isn’t always what one expects when starting a new project. When I created mongo-php-adapter a few years back, I had only one use-case in mind: I needed to buy time to rewrite the entire data access layer of the Doctrine MongoDB ODM to use the new MongoDB driver. PHP 7 was the hot new thing, and our users couldn’t upgrade because of the MongoDB ODM using an old, deprecated driver. To quickly ease the pain, I spent my Christmas break writing an adapter that provided the legacy API built on top of the new MongoDB driver. With this adapter, people could continue using the ODM, upgrade to PHP 7, all while I was trying to find the time to make the necessary changes to the ODM itself.

As it turns out, it wasn’t just the Doctrine MongoDB ODM that suffered from the driver problem: many other applications built on top of the legacy MongoDB driver suddenly needed a sizeable effort to be updated to the new driver. For a lot of engineers, this library was exactly what they needed: they could move their applications to PHP 7, use a supported driver, and didn’t have to justify to their boss why they need to almost rewrite the entire data access layer. Like many other Open Source engineers, I never thought that so many people would start using something I built to solve a problem I thought only I had.

I was in a lucky position: there weren’t many bug reports, as the scope of the library was pretty limited and didn’t contain all that much logic. When there were bugs, I was able to justify working on them during my paid work hours, as we were using MongoDB ODM and the adapter library ourselves. My boss and I had an agreement that I get to spend a certain amount of time on the Open Source libraries that I like to maintain, especially if they were used at the company. This extended to other engineers as well: anyone was encouraged to work on Open Source projects. Few people chose to do so, but for those that wanted to, being allowed to take a few hours of paid work time for their passion projects was a great deal. So, that’s how we can make Open Source sustainable. But is it really the solution to the problem?

Making Software Engineering Sustainable

When I started my last job, it was with people that I have known for 6 years. In fact, we’ve worked together as a team at one company, then collectively moved to a newly-formed company to build an e-commerce application on a green field. It was the dream come true. While we talked details, I was pretty clear about my Open Source responsibilities, and that I’d like to be able to work on Open Source projects during paid hours. It was my way of ensuring that the company gives back to Open Source. My boss was fine with this, especially since we planned to use the libraries I maintain. The people funding the company were ok with this, because they knew that it was the only way to get the entire engineering team. Everybody saw it as a win for them, the company, and the ecosystem.

The contract was typical for an IT job in Munich and most likely Germany: 40-hour work week, overtime is not paid. We got 25 days of paid time off, which was more than the 20 days required by law, but less than the 30 days usually given to engineers. The pay was a little higher than average, but not unreasonably high. Combined with the ability to work on Open Source projects and finally getting to build software on a green field, this seemed like an absolute win to me as well. Spoiler alert: it wasn’t.

Before I go into details, I’d like to add a short disclaimer: this is my personal story of how this setup wasn’t sustainable for me. I am very well aware that the above contract is very good compared to what is standard in other companies or other countries. Yet it is the only example I’ve personally experienced, so it’s the only one I can use to show why even this isn’t sustainable. I’d love to hear stories of people in similar or completely different situations, and to get different points of view on this issue. The problems I outline here are my personal problems, and yours may vary. Some people will see my problems as minor compared to theirs, and that is fine. Maybe they are minor problems, but they are still problems for me.

I mentioned that my contract specifies that overtime isn’t paid. This is normal in most IT jobs here, and most candidates never raise a point about this. After all, we love our flexible hours, right? You get to work when is convenient for you. If you need to leave early to see a doctor, go and leave early. If you need to finish something, you stay late to finish it or do it on the weekend if it’s really important. What I’ve learned over the years is that while this is what you have in mind, it’s not always what your boss has in mind. Case in point: I wanted to leave work early twice a week to pick my son up from daycare and spend time with him. When I suggested to my boss that I’d like to leave an hour earlier and then work from home in the evening to make sure my work gets done, he wasn’t entirely convinced of the idea. I started doing this once a week instead of twice and enjoyed every second I got to spend with my son: we’d walk through the park, enjoy a snack by the lake, and just unwind after a hard day of work. It was perfect for us. It was perfect right up to the moment where my boss told me that it’s a bit inconvenient because “when somebody leaves early, it demotivates the rest of the team”. Wow. I later found out that my boss himself came under fire from his bosses because the engineering office was “deserted at 5:30pm”.

I ignored this and kept doing my thing. Every Wednesday, I’d leave at 3:30pm, work another half hour in the train on the way home, then spend some time with my son. Why would I change something? I normally started working at 7am, so even accounting for an hour of lunch break, I accumulated 8 hours that day, and I haven’t even done my “work from home” thing to catch up on work that happened after I left. Still, it left a bad taste in my mouth. This was my employer telling me that they weren’t happy with me taking time for my family, time for myself. It was important to me, but they didn’t care.

As always in tech, there were times when we had to get things done by a very ambitious deadline. But while the contract says that salary covers any and all overtime and we aren’t entitled to extra pay, this isn’t the entire truth. German law sets a lot of rules on how much overtime can be required, and how an employee has to be compensated for it. I’m not a lawyer so I’ll keep this simple and incomplete. In a nutshell, if your employer just expects you to work overtime, you can ignore them. An employer can mandate you to work overtime, but there are limits as to when they can do so, for how long, and how much overtime they can require. Many people see this as the boring part of labour law, but it’s an essential part. By law, you may not work more than 48 hours per week. This can be increased to 60 hours for short periods of time, but has to be settled within 6 months. When I say “settled”, I don’t mean “compensated”: you’re entitled to time off. German law knows that while money can buy you a lot of things, time isn’t one of them. Same goes for weekends: if you work on a Sunday, you’re entitled to a full day off within 2 weeks.

Employers count on people not knowing this, and they get away with it. Many people currently working in IT are young. They don’t have families and enjoy working in a fast-paced environment with lots of challenges and likeminded people. So when our bosses asked us to do overtime, most people were perfectly fine with it. Some employees even previously lobbied for the team to work more hours because they like it “when the team is in the office on a Friday night to make the sprint goal happen”. I may be getting old, but I’d rather sit at home reading my sons a good-night story before bringing them to bed than sitting in the office at 9pm on a Friday night.

Getting “Paid” to Work on Open Source

Why am I saying this? While I was able to work on Open Source projects during paid hours, I could only take 2-3 hours every week to do so. Most of this was done on Friday mornings, when my wife worked from home and I didn’t have to bring the kids to daycare. I left home at 6am, sat down at my favourite coffee place and worked on Doctrine projects for 90 minutes before going to the office. The rest of those weekly 2-3 hours was spent looking at issues as my GitHub notifications filled up during the week (whenever I have an unread notification, I can’t not look at it – it’s a big stress factor for me). This wasn’t time spent on some random project that played a minor role in our tech stack. This was the back bone of our business: storing data in the database and retrieving it. Without my work and the work of my co-maintainers, we wouldn’t have been able to build our product this quickly.

Over my 3 years at the company, I averaged around 46 hours a week. Due to the language in the contract, any overtime was immediately “lost” and compensated for. I was able to shift around some of those hours (e.g. by leaving early once in a while), but it’s not like I was able to work 46 hours in one week and then recoup those hours by not coming in for a day the week after. Since my employer paid me for 40 hours a week, I effectively only worked on Open Source during my free time. I kept telling myself that I get paid to work on Open Source, but I didn’t.

Change is necessary

You may be wondering why I’ve written about this very personal story. The truth is that when I got the opportunity to join MongoDB and left my company, things turned sour quickly. I am currently in a legal battle against my previous employer over unpaid bonuses. I’ve also been accused of having neglected my actual job to work on Open Source software, giving away intellectual property to competitors, and only furthering my own career. I have been struggling with depression before, and this episode brought me to the point where I was contemplating suicide. I am currently getting treatment for my medical health and I am progressing slowly. But every time I get a letter from my lawyer or from the court, I notice that I’m not there yet. That’s what I mean when I say we need to make our jobs more sustainable.

I’m 35 years old, I’ve been writing programs since I was 14. I started working with PHP 15 years ago, and it’s been my job for the past 11 years. In these 11 years, MongoDB is now my 8th employer. I’ve quit one job on the second day of employment, and I’ve had to quit one job before I started because my mental health at the time didn’t allow me to work. My last three employments have taken up 7 of these 11 years, so it has settled a little bit. This isn’t just me: most of my coworkers in previous jobs have similar numbers. I’ve seen people come and go at a record pace.

I’m 35, and the retirement age in Germany is currently 67. I have to work for almost 32 more years before I can retire with full benefits, whatever those will be when it’s time. If the previous trend continues, I’ll have worked for more than 30 companies by then. But it isn’t just that. The oldest candidate I interviewed was 39. The oldest coworker I had ever was younger than 50. Can you imagine being 60 years old and looking for a job in our industry? Can you imagine interviewing with a person who is significantly younger than your children? On the other hand, can you see yourself interviewing a 60-year old candidate? Would you give them a fair chance, or would you rather hire a young, dynamic, 20-something person who’ll happily work 50 hours or more because they don’t have a family waiting for them at home? A person who argues to work on the weekends to get something done because they don’t know that the employer is legally required to give them a day off in return?

With my new job at MongoDB, I’m now able to make some changes. I can work from home (or any other decently equipped office for that matter), and since my team is in New York City 6 time zones behind me, my working hours don’t play that much of a role. This way I’m able to start working early and get out early to spend time with the family. I can also take a break during the day and work late, especially if I have meetings to attend or if I want to pair-program with someone on my team. The flexibility gives me a certain peace of mind, and it increases my productivity. I’m also allowed to spend more time working on the Doctrine MongoDB ODM, going as far as scheduling it in our issue tracker to ensure the driver change mentioned above finally gets done.

So while we definitely need to make Open Source sustainable, we also need to start making our jobs, and our industry more sustainable. It surprises me that in an industry where all we need to work is a computer and a working internet connection, we still require teams to be in one location at the same time, every day. It surprises me that many companies and many bosses have an issue with their employees taking time for themselves or their families.

It is on us to change this. Go home at 3pm once a week to do whatever you want. Work from home regularly and focus on your mental wellbeing. If you are a manager, give your employees the freedom to do these things, and allow yourself the same freedoms. Go ask about working part-time if you can afford it financially. Work 35 hours a week or even 30, and spend the new-found time to do the things that make you happy. You’ll notice that it not only improves your productivity and your happiness, but it also makes it normal to put our lives and health first, instead of focussing only on our job. After all, life’s too short to only enjoy it after retirement.

Changing Lanes After Freelancing for a Decade

PHP is the first server-side programming language I learned. My story might be very familiar to yours: after I built a few static websites with HTML, CSS, and JavaScript, I was trying to add a contact form to my sites, then a discussion board, then a news system, etc. By the time I had two major languages to choose from: ASP and PHP. I picked up the later because of the “Open Source” tag on it. Of course, I knew nothing about open source then, I just thought I could get all the software I need for FREE and that made perfect sense to pick that side.

I labeled myself as a PHP developer for a while and started to meet small business owners that wanted a website for lower costs, and they’ve heard enough good things like PHP, MySQL, Apache, and Linux that you can get for free. In the beginning, it’s just doing a favor for a friend’s friend and then it turned into a freelancing business. Then the WordPress era came and I switched to be a WordPress developer and happily migrated my clients’ sites to that platform.

In my story plot, I never thought to seek a full-time job as a PHP or WordPress developer and it’s just natural for me to be a freelancer because the open source ecosystem empowers me in a highly affordable way that the only cost is almost only my time. So that’s pretty much what my career was about in the past decade, and I will always be grateful to the PHP, WordPress and Open Source communities that provide me a prospering life.

The unique WordPress freelancer community

My business was not always on sunny days, especially when I started to realize that doing my work with open source software didn’t mean I couldn’t charge a premium rate. I kept increasing my rates to test the local market but it didn’t respond to it well. So I started to look for web development gigs in the U.S. or Europe (I am based in Taiwan).

To get more global clients I applied for Codeable and got accepted in 2017. Codeable is the only freelancing market designed for outsourcing WordPress projects and it created a totally different freelancing experience for me. As the New York Times bestselling author, Paul Jarvis, always says to freelancers, “work for yourself doesn’t mean you have to work by yourself”, a freelancer community is actually very crucial for our emotional wellbeing. When I joined Codeable, I suddenly had like 100 coworkers whom I could talk to in the virtual office space (Slack) about WordPress and business. You also get a top-notch support team to back you up when you’re in need. From there I found myself actually have been longing for a professional network that I haven’t had for quite a long time.

The uniqueness of Codeable is that experts (developers) on the platform are collaborating together instead of competing with each other. It may not make sense that why are we willing to help each other since the client would only hire one of us and you surely don’t get a share for helping the person who wins the project. However, it’s the culture carefully developed that way so once you’ve been there and gotten helped by your fellow experts, you have role models to look up to and follow their steps.

The right community channeled me with more opportunities

I also benefited from Codeable for landing high-quality clients that pay decently and run their businesses successfully with WordPress. To be precise, I was getting more and more work about Gravity Forms, the leading premium form builder in WordPress which works in a developer-friendly way to extend its functionality. I have been using Gravity Forms for a couple of years for client projects but I rarely added any customization. Working with clients on Codeable (including Steve Henty, the founder of Gravity Flow, who is also the lead developer of Gravity Forms) is eye-opening that I had the first-hand experience to validate how one plugin could have its own ecosystem (set aside from the WordPress ecosystem) and how its partners are profiting from it.

With so many inspiring experiences happened consecutively to me, it totally changed my mind and view about businesses running with open source software. I saw different business models and found a spot that I could fit in perfectly. The joy of fulfillment that something I contributed would live long in the software and benefit its users, makes me think about a different job role in my career. And the image was getting more and more vivid that I couldn’t resist.

In October 2018 I got the offer of a full-time developer position in Rocket Genius, the parent company of Gravity Forms and started work in January 2019. Jotting down this brief history of my career is not only as a self-reflection at the end of a year, but I also wanted to leave something for latecomers to see a different career path that is less talked about.

For me personally, when considering such a move, I was still struggling despite this was something I asked for deep inside my heart. The struggle came from a self-identity shift from a freelancer to an employee. I couldn’t help but feel guilty to say goodbye to my old self. However, I learned to see it as the “growing pains” that we human beings always have when we’re teenagers. Once you overcome that pain, you’re a better version of yourself.

A full-time position that equips me with more resources

After joining Rocket Genius I get more resources in improving my development skills. In the past year, the team has added a lot of accessibility and security improvements to our products. Through the process, we’ve learned together from our consultants. While these topics are not PHP specific, I’m sure they are as important as creating new features in the web industry that developers should all spend time to dive into.

To deliver stable products we also add new tests (both unit tests and acceptance tests) when applicable. I have to admit that these are something I’ve always wanted to invest more time in but failed when I ran my own freelancing business. The mindset of doing outsourcing projects is totally different that if your clients are not willing to spend time and money in accessibility, security and automated tests, even if you are fully aware of the importance of such matters, things you can do in the projects are very limited. It’s not that freelancers can only deliver inaccessible, insecure or unstable results, I’m pretty sure a lot of my beloved freelancer fellows are doing much better than me. But in my case, these are really positive points why I chose to work for Rocket Genius after more than a decade of profiting freelancing.

Things are changing from time to time just like how PHP evolves along the way! Changing isn’t always just about being better or worse, I believe if we could properly document the difference, we can learn from history and help newcomers. So that’s what this little story is about! I’m also looking forward to hearing from you if you have something to share. I hope this tiny piece can bring some fresh perspectives for you this holiday season!

If You Build It, They Might Not Come

Ten years ago, I was working at a small web development firm across the street from Arizona State University. We worked with several startup customers attempting to follow their dreams, create something amazing, and make lots of money! Some of my favorite times at Synapse Studios involved sitting down with new customers in our initial consultations to get an understanding of what it was that they wanted our help in creating. Sometimes, the ideas that were pitched sounded terrible, and instead of being a bunch of money-grabbing yes-men, we had to deliver the bad news: “Sure, we can absolutely build something like that, but we don’t think it’s going to be worth your money.”

This can be hard news to hear and accept for eager entrepreneurs. Often excitement and/or pride blinds us to the reality that only a small minority of startups succeed, especially those run by first-time entrepreneurs. For those that do, most of them have to pivot—often multiple times—from their original ideas and plans. I can’t imagine the devastation for those who put many thousands of dollars into their dream, only to watch it fail. And the kind of failure that stings the most is not due to technical limitations, bad business practices, or intense competition, it’s being overlooked or ignored.

There is a famous quote, “If you build it, they will come,” from the classic, baseball-related U.S. film: Field of Dreams. Despite the line being quoted out of context and incorrectly—it’s actually “he”, not “they”—it has stuck around and found itself inevitably intertwined with naïve ideas about the “American Dream” and entrepreneurship, in general. Unfortunately, the harsh reality is that you can build something, even something really amazing, and they might not come.

This disheartening truth is not just a precedent for the startup scene. I spent about four years in Seattle working at Amazon Web Services in a department known Developer Resources. My team created the SDKs that programmers use to interact with the AWS service APIs from their favorite programming languages. It was there I that I really started getting into open source, as the AWS SDK for PHP was presented as an open source library to customers. Also, I spent time contributing to other projects like Guzzle and PSR standards.

Part of our time at work was spent responding to support requests, forum posts, GitHub issues, and Stack Overflow posts in order to help customers using AWS services through our tools. I found the most frustrating moments were those in which customers asked about things that could have been easily solved with code and documentation that we had already published. “I already built this, why aren’t they using it?” I would say to myself.

Creating something is not the same as advertising or marketing it. Not all published works are automatically broadcast into every human brain. We cannot assume that just because something exists, that everyone knows where it is, has seen it, and understands it completely. Especially when we are the creator, we should not expect everyone to have the same level of intimate knowledge about our creation as we do or the same amount of passion or interest.

Although it may be tempting to respond with the impatient and insulting “RTFM”, doing so is not only disrespectful, it is also a great disservice to ourselves for at least two reasons. First, it robs us of the opportunity of making a human connection. It turned out for me that supporting the software I was writing was a great way to network, make friends, and make people smile. Secondly, questions about our creations can help us gain different perspectives about them. New features, new content, and new ideas come from listening to our clients, customers, and competitors, and discovering better ways to help and serve others. Even when detractors say things that upset us, we can still look for truth behind the emotions that we can use to improve.

It’s also hard to face rejection when we attempt to contribute to someone else’s creation. Contributing to others’ open source projects can be frustrating when they do not accept or use our contributions. When this happens, we should not assume ill intent. In most cases, open source maintainers have decided to do something in a slightly different way or timeline or may have just overlooked something.

It can happen at work too. I’ve built things—really cool things—in my current position at McGraw Hill that have gone unused. Sometimes, this is because the direction of the technologies or priorities has shifted, based on new business requirements. Other times, it’s just because people are busy. Yes, sometimes other people are just too busy to care about every little thing we do. If I didn’t make it convenient enough for them to inject it into their busy schedule, it’s not necessarily their fault.

It’s also often the case that our invitations are more narrowly-scoped that we might initially think. Just because it’s in the company wiki, does not mean everyone has read it or can even find it. Not everyone will notice our posts in Slack (or Twitter). We will need to explicitly invite others. Sometimes we’ll need to sit down with people one-on-one or in small groups. Sometimes we may need to present something to the CTO and get their help to create a top-down initiative. Sometimes we’ll need to pay for advertising. Overcoming the “they might not come” phase may require even more work than the “if you build it” phase.

When our creations don’t get the attention we want or expect, we must let go of the negative emotions that often arise. We cannot give into the temptation to blame others, complain, or proclaim, “That’s not fair!” However, it doesn’t mean we should give up on ourselves or our creations. When we strip away our self-centered feelings of entitlement, we must replace it with empathy towards others, not apathy.

When we build anything, we should be approaching solutions from the perspective of our customers. Fields like marketing, technical evangelism, and user experience (UX) design have all come about to help creators better connect to their customers. When we give people what they truly want or need, and they give us money, recognition, or satisfaction in return, that’s a win-win. We just can’t expect to always get it right, especially the first time, with our own limited perspectives.

Let’s move past the frustration of failure and rejection and keep learning and building. Let’s move forward with an understanding that even though we might build something, they might not come. Let’s try and meet them where they are instead. Let’s listen to and empathize with them, and they might stay with us a while. Let’s truly be agile.

I <3 PHP (or is it PHP <3 me? ... needle, haystack?)

There was a popular article a few years back calling PHP a “fractal of bad design.” It was basically a re-telling of the things we all know that are frustrating about PHP, but it gained a huge amount of popularity especially in the spaces where people already think of PHP as a bad language.

Thankfully, these days, that post matters only as a piece of back story to explain the name Phil Sturgeon used for a now-extremely popular package, “Fractal”: purely so it could bump that article down in the Google rankings.

These days, if you Google “PHP fractal”, that article is the third down, and almost all of the remaining top 10 results reference Phil’s package—or his original article about it.

I love this story—because I think Phil is hilarious and clever.

I also love this story because I think that PHP does have some serious drawbacks, many of which are detailed in that article, but I also think that the truest type of love—just like love of your family members, love of the annoying-but-endearing member of your friend group, or love of your country—requires that you both acknowledge its faults and then love it nonetheless. “Dissent is the highest form of patriotism.” “The Church is a whore, but she’s my mother.”

Last week I was on a podcast (How To Code Well Podcast – Why Is Laravel So Good!) in which the host laughingly asked, “wait, you were moving to Ruby but you came back? Why?”

It’s easy to get defensive when you’ve aligned yourself with a thing. “I’m a PHP programmer and someone implied it isn’t a joy to write with; I’m big mad!” But the thing is, first of all, I’m a programmer who happens to be writing PHP these days. PHP doesn’t define me. And second of all, if you think PHP is the most enjoyable language to write purely based on syntax then you likely haven’t written in other languages like Ruby.

Anyone who’s worked with Ruby knows it’s a joy to write; PHP is a lot of things, but syntactically beautiful is not one of them. Ruby is like the slick new sports car that you drove around at the dealership once, while PHP is more like your ten-year-old Corolla back in the garage.

In fact, I constantly describe my attitude toward PHP by saying “I want to write PHP like it’s Ruby.”

So… why don’t I just write Ruby? Why work with PHP at all? Turns out… it actually has a lot going for it.

I can deploy PHP anywhere I want

Have you ever tried to host a Ruby app yourself? Have you ever paid a Heroku bill? Have you ever tried running .NET on your own server? Have you ever tried to build a full stack Node application?

PHP is everywhere and it can go anywhere. It’s cheap and easy to host. No other language has that same level of ubiquity—not even JavaScript.

I can write PHP however I want

Many other languages tout the fact that there is a single canonical way to write them—to the point when some actually have formatters built into the language itself. I don’t hate that idea, but I do have one concern: if the people working in the core of PHP today were to enforce a standard PHP syntax, I wouldn’t want to write it anymore, because I don’t want to write PHP the way they do.

But… because PHP is so flexible, it doesn’t matter. I’m going to keep writing my Ruby-esque PHP, concise and absent of docblocks and return type hints; and they’ll write their Java-esque PHP, all verbose and enterprise-y; and most importantly, we each will benefit from the others’ contributions because we’re working in the same language and ecosystem.

PHP is a dynamic language. It’s duck-typed. Its type system is entirely optional. There is room for me and room for them and room for everyone else in-between and PHP just keeps chugging along.

There is a lot I can do, easily, with PHP

I don’t have to compile PHP. I can write a simple script that’s just a few lines and run it right in PHP. I can run it from my console, throw a single file up on a server, build a complex custom full stack web app, easily build CMSes and APIs and forums and wikis and blogs and admin panels and shopping carts and much, much more.

The ecosystem of tooling is incredible. No other language has this many packages available to make it possible for you to do this many things.

You can make money as a PHP developer almost immediately

I’ve helped many friends go from no programming chops to making money as a programmer. And, despite how many people I’ve seen able to get good jobs coming out of full-stack Node or Ruby bootcamps, there are many more I’ve seen completely unable to find work. But when I teach someone a little bit of HTML, CSS, JavaScript, and PHP, they’re immediately employable by WordPress shops and small businesses that need WordPress web sites.

Developing for WordPress is a completely acceptable long-term occupation—but it’s also a super easy introduction into the world of building web sites with PHP, and a space where someone can make money as they choose to transition to some other sector of PHP. Again, this is relatively unique to PHP, and I love it.

Turns out, it’s even great for custom web apps

You might be tempted to think that I’m saying PHP works well for CMSes and Wikis, but that other languages and tools win out for custom web application development.

But even there—with competition like Ruby on Rails, full-stack Node, ASP.NET, and Elixir—PHP shines. Just look at the list of tools and services and pre-built components you get when you’re working with Laravel and Symfony. How many rich communities we have around full-stack web application development frameworks—not just one, or two, but at least five major web frameworks (Laravel, Symfony, Cake, CodeIgniter, and Yii).

I could go on for days about the incredibly rich and robust ecosystem of tools, training, platforms, and services that support a web application developer who wants to work in PHP, but I won’t.

Let’s just leave it here: there is so much we can do with PHP. It has warts, for sure. I don’t care if the parameter order of its native functions are internally consistent with string methods vs. array methods; they’re still confusing as hell. But, in the end, that’s a minor annoyance. Yes, I wish my code could be more terse… but we did just get single-line closures! Baby steps!

And more importantly than that… syntax is not all that makes a language. Its communities do. Its tooling. Its supporting services. Its frameworks and components. Its education. Its ease-of-learning. Its ease-of-deploying. Its flexibility.

So, there it is. PHP is my annoying-but-lovable friend. Acknowledging its constraints doesn’t make me love it less; it makes my love for it more real. I’m not sticking my head in the ground and pretending I can’t hear the criticisms. Instead, I share them… and that doesn’t diminish my appreciation for the Programming Language With the Stupidest Acronym Of All, PHP: Hypertext Preprocessor.

Passion of the Open Sourcerer

Hello, my friend. Stay awhile and listen.

10 Years ago

I was sitting behind a regular office desk, perhaps the same type
as yours, as you are reading this, when I had the intention of documenting a Symfony 1 application I was working on.

I had freshly installed this incredible tool I had just learned about using PEAR and read up on how to use it. This was my first interaction with phpDocumentor. At that point in my professional life, it felt exciting to see Open Source applications do their magic, and this time was nothing different for me. I typed the commands that the documentation prescribed and hit the Enter key; anticipation built up while the first pieces of output darted across my screen. My typed incantation was working!

Never lose the feeling of wonder and magic; too many things in life are boring enough.

Unfortunately, my excitement did not last long for the fans on my laptop started to spin loudly. The darting output slowed down to the point where I could write faster and my O/S was alarming me that its memory was overflowing. My computer had become unusable and I was left without any documentation still.

At that point I was frustrated, the magic had gone and only an empty feeling remained. But I still believed. And as such I set out to contribute to this project that I believed in; I read its source, followed its logic and undertook to patch its core so that it would use less memory.

When you are confronted with a setback; turn the table and grow from that situation; if you can do that in a way that benefits others too, even better.

In the end, I failed. As I recall, I found the core architecture unsuitable for low memory consumption and in my youthful overconfidence I believed I could do better and started a new Open Source project called DocBlox. My aim with DocBlox was to do the same thing but ‘better’. I set out to create an architecture where memory consumption was low, speeds were high and I could feel the magic again. With that magic, my passion grew as I was creating something new of which I felt that it would benefit others like me.

At some point, I discovered that the original creators of phpDocumentor were unable to actively maintain the project. I felt uneasy in my decision to start a competing project while that one was struggling. This was the first time I discovered how few active contributors well-known projects frequently have. So I decided to reach out to Chuck Burgess, one of the core developers on phpDocumentor, to discuss whether DocBlox might instead merge with them and give birth to a new phpDocumentor: version 2. He was excited about the prospect and not long thereafter we merged the projects together and phpDocumentor 2 was a fact; arisen anew like a phoenix!

As a community we stand stronger; Open Source projects count few consistently active contributors; often it is more productive to work together than to work against one another.

Now, 10 years later

Jaap and I, the two active core contributors on the project at the time of writing, have spent many hours of our free time to get phpDocumentor 3 out there. We have dreams and initiatives to allow it to generate even more relevant documentation, including reference documentation.

As I reflect on how this all started, there are moments where it baffles me that the project is still active and improving after 10 years. I have seen contributors come and go, ideas and initiatives started, aborted and finished. But one thing was consistent:

The project is carried by the burning passion of a few.

Now, I can see that almost every Open Source project is carried by the passion of a few, often just one or two people. Some of these people have been active for a decade or more. This includes well-known and well-used projects like PHPUnit or Composer but also outside of PHP; for example, cURL and OpenSSL.

Are you one of these few? You are a magnificent hero and I thank you from the bottom of my heart.

Are you not? Don’t fret! If you can, pick a project and help out; even with just one pull request. You cannot imagine how much that means to many a maintainer.

Ever since I picked up my passion for phpDocumentor, I have also seen periods of stress, I have been burned out and sometimes even felt desperation or anxiety. I wondered whether working on phpDocumentor mattered, whether it really helps people. And sometimes, I still do. During these periods the passion is lower, and I am not able to put as much effort in the project as I would have liked.

If you have ever felt like this: it is OK. Let it be for a while, do what you need to re-energize and talk to your co-maintainers about it. Then, after a while, agree to meet somewhere in person and just talk about your project. If you are in a good place again, you will feel engaged and that passion flares up.

At the same time, I have felt great pride and felt humbled that so many
use phpDocumentor or its components. The community, both those who contribute and those who use phpDocumentor, have been a source of passion for making something worthwhile. The numerous talks there have been at conferences, online or just through a simple tweet; all of this keeps the candle burning and even reignites it again when it’s needed.

And then

Now, I want to thank those who went on this journey with me for all these years and for the years to come. And I hope, that if you didn’t know already, that you can feel that people working on Open Source need you to keep going. Your support, in whichever way you can, is vital to keep those few going who maintain these projects.

You can make a difference.

Applying Clean Code…(Realistically)

It is no secret that when tasked with solving problems in the real-world our solutions will always be just that, less than ideal. With that in mind, how can we make sure that we execute our solutions as cleanly as possible? How can we realistically ensure that any of our theoretical and idealistic approaches to problems can withstand the ever-growing demands of the workplace when applied?

The answer: clean code – applied realistically.

But what does it mean to realistically apply clean code? And what is clean code even really?

If you aren’t already googling away, I’ll make your life easy by summarizing both questions into one simple phrase, “code that humans can read, and understand.”

As they say winners know when to stop. If you often find yourself getting carried away with over-engineered, but admirably elegant architectures – then stop! There is beauty in simplicity; below we expand upon this simple phrase and what exactly it entails.

You Show Intent

The naming of things such as variables, methods and classes shows the intention of your logic.

You Are Consistent

There exists a clear naming convention – PSR-12 anyone?

You Reuse Code

Where applicable and where the effort required is low, you write generic and reusable code to simplify your solutions and reduce the ability to introduce new bugs.

You Don’t Comment Bad Code

Understanding what the code is doing is not equivalent to the code being good. If you have to write a comment, chances are, you are better off rewriting your solution in a format that doesn’t require comments.

You Don’t Pre-optimise

Yes, you read me right the first time. We often tend to pre-optimise code when the scale at which it would operate doesn’t even concern us yet, tone it down a bit. There is such a thing as coding too defensively, but that’s a topic for another discussion. Keep it current; keep it simple. We’ll cross that bridge when we get there.

I know it sounds easier said than done, but just focusing on these 5 points can save you a bunch of headaches down the line. That being said, enjoy the festivities and remember: keep it simple!

Dealing With Developer Overload

Being a developer and being prone to anxiety aren’t what you would consider to be a good mix. Even if a developer isn’t inclined you can certainly understand why today’s web development environment would make any sane person’s head spin. Even “experienced” developers can get dizzy at times figuring out which paths to take forward or even if they are taking the right path right now.

Remember This? Pepperidge Farm Remembers.

I would like to share my first tech stack knowing it will date me: HTML-2 and CSS. That’s it. A few years later you can add JavaScript and Flash to that list, but that was basically it in terms of front-end development. Classic ASP, PHP, and Coldfusion existed but there wasn’t pressure to learn these to publish web pages.

Today? Just take a look at this roadmap for becoming a web developer in 2019. Things certainly have changed.

Overload is Real

Developer Overload is real and I believe every developer has or will experience it (myself included). But what we have termed as Developer Overload now isn’t just simply a reaction to the complex world of web development. The simple fact that the world is a complex place and with it development is acceptable. But let’s consider three root causes that developers might have in their control:

Job Security (“What do I need to know to still have a job in 2 years?”) – Most of us have experienced Flash in its heyday but they aren’t as many Actionscript developers as there was 5 years ago. Most of us are stressed, but hopefully we like what we do. So we would like to keep on doing it.

Deadlines (“I have a deadline and I need to (re)learn this fast.”) – As developers, it’s common in our industry to “learn on the fly” under pressure. How many of us started working on a PHP or JavaScript project and then we run into something that requires us to go into Google and pull up a tutorial?

Our Natural Curiosity – The tendency is within us. We are explorers. Tinkerers. We hunger of new and more efficient things.

Any combination of these is something that many developers experience constantly. The effects are what you would expect from burnout and even certain types of depression: a lesser desire to be naturally curious, negative emotions, and not enjoying your job or work.

How To Deal

That is, after all, the title of this post. One can go into great length on each of these points but here are the highlights:

Focus

Because of the enormous amount of information out there, focus is one of the most valuable skills you can master as a developer. As developers we often bite off more than we can chew…we also tend to take that bite from the wrong side of the sandwich.

Focus isn’t about staying with the same platform or tech. It’s about determining what type of work gives you the most joy. This will require you to pick your battles and don’t be distracted by the next shiny object or what could be the next “big thing”.

What comes along with focus is checking your ego: developers are obsessed with the notion of “best practices” and using the latest and greatest techniques. But often we do NOT have to reinvent the wheel. That next blog you need to build may NOT need to be a static JAMstack setup – plain HTML or WordPress (or your CMS of course) but me just fine. Keeping things simple helps you focus on the more important things.

Organization

Organization means less surprises and having a plan. Use whatever apps or old school methods (bullet journals!) help you get more organized and you’ll feel more control in your life. Whatever the method find what works for you and stick to it. Include in your persona roadmaps a set amount of hours in a week to sit down and work on a side project, learn a new skill, or read.

Another great point that took me a while to grasp: learning more about keeping our code organized and well written is a good return on investment because it saves you time down the road.

Self-Care

We are no good to ourselves, our work, and our families if we have constant dread and are drowning in our work. Poor mental and physical health is an enemy of dealing with overload just like having system resources bogging down your IDE or compiler on your machine is an enemy to your dev time.

Physically make sure you are getting enough rest, you are paying attention to what you are eating, and think about your long-term health (do something about that desk or chair that you know is bothering your body). Mental health is also vital – many developers have to learn the hard way to take a break (from tech or just social media) and make sure they are getting the right amount and kind of human interaction (and talking with other developers about coding isn’t completely what I’m talking about).

Remember: It’s ok to break code. But it’s not ok to break yourself.

You Can Do It!

This post is only a fraction of the subject I talk with other developers about. But it’s a nice set of reminders to remember that Developer Overland is real. There is no real “cure” outside of leaving the industry all together. But realize that development is getting more complex by the day… a fact that recently many developers are pushing back against. There are a lot of pressures coming from work, family, and yourself – and it’s important to remember to apply self-care where you need it.

Someday we’ll be able to tell young ones about the golden age of web development and that we managed to rise above the challenges while maintaining our sanity.

Growing Gradually in PHP

With the release of the much-anticipated PHP 7.4 and the planned release of PHP 8.0 next year, there’s been discussion in some quarters of whether or not PHP is getting “too strict”. Some have even phrased it as an existential split between two fundamentally different views of the language, incompatible worldviews on strictness vs flexibility that are at odds and will tear the language apart bit by bit if nothing is done to address that schism.

I do not share that view. Quite the contrary, I find PHP’s balance in recent years between casual code style and more formal code style to be one of its most remarkable and strongest features, and one we should be both proud of and continue to encourage.

It’s true there really are different ways and styles to approach programming; not all of them are always valid, but most are valid in certain situations and contexts. When writing a one-off simple script, or a hit counter, or some other “casual” use case where the code base is small, it’s entirely true that a lot of the formalisms and stricter, more pedantic language features are unnecessary and can in some cases, get in the way.

On the other hand, the larger and more complex your application, the more that formalism and strictness can save your butt. Rather than get in your way, it heads off bugs and design flaws early, before they can fester and become even more expensive to fix. I wouldn’t care if a simple side project app isn’t strictly typed all the way through; I wouldn’t trust an ecommerce application that isn’t.

But that’s the great thing about PHP: We can do both. While I encourage all developers to leverage PHP’s opt-in strictness and typing and other formal design tools as much as possible, they’re not required. If your use case really doesn’t call for them… don’t use them. If it does, use them.

Where some languages are engineered for big systems and thus have a high ramp-up to learn all the moving parts, and others are engineered for casual use cases and thus struggle to be scalable to large and complex use cases, PHP allows a “pretty good” of both worlds. It’s easily approachable, easy to learn, but contains an increasing number of features to allow for a more robust and self-checking programming style.

We can (and frequently do) debate when using those stricter, more explicit features is appropriate; personally I prefer to use them even on tiny projects as it’s good practice, but not everyone does, and that’s OK. But the fact that PHP lets you gradually scale from untyped loosey-goosey code to partially typed to strictly typed code, with casual error handling to strict error handling, all in the same language, all in an inter-compatible way, is absolutely incredible and something the PHP core team should be proud of.

I’m all for adding more robust formal tools to the language; I’m excited about typed properties and short lambdas in PHP 7.4; I am looking forward to union types in PHP 8, and I’d be ecstatic if we could figure out how to wrap intersection types and generics into the mix. There’s plenty of other language tools to help constrain large code bases and algorithms we could consider, too. But by necessity all of those would be opt-in features, just like types are now; and that’s what’s great about them, and about PHP.

I’ve never agreed with the argument that type definitions or classes make the language inherently harder to learn, but PHP, fairly unique among languages I know, lets you learn them piece by piece or all at once at whatever your pace happens to be. That wasn’t a conscious, deliberate decision, but it’s how the language has evolved and we are all better off for it.

We don’t need to split up PHP. We don’t need divergent “modes” or “editions”. We need to continue doing what PHP has been so remarkably good at: building a language that scales with you, from casual loosey-goosey scripts to formally typed and structured and error checked designs that would make a Computer Science professor proud. Fitting all of those into one language is hard, but PHP has done it, and we should continue to do it.

And that’s something we can all be thankful for.

Thinking about Dependency Management

The last 8 years can be characterized as a massive change in how we work with dependencies within PHP projects. The arrival and further advancement of Composer has brought a big change in how we decide which dependencies to use, how to maintain them and how we update and upgrade them. Only few people would deem this change as not positive – overall this has been a big success story for the PHP community and environment.

Unfortunately, often each blessing comes with strings attached, and this is no exception. Because it is so easy to add another dependency, we tend to add just another one without thinking about whether we really need it or if there isn’t another solution available. You may not maintain the library that you just added as a dependency, but the moment you add it it becomes your liability. Nowadays, it’s often not simply adding a single library – those have dependencies as well, and become part of your project and your liability as well. It may not be as bad as in the JavaScript world with npm (1, 2), but are we always really sure we want to have all those dependencies?

While we all enjoyed the new freedom made possible with Composer and earned the fruits, I think we didn’t look into the other non-positive aspects as much as we should. Are you sure you know about all packages that are in your application? Not just the direct ones you declared as a dependency, but the dependencies of those, and the dependencies of the dependencies of your dependencies? You could prolong this chain even more I guess.

The issue here is trust. For example, if you decide to add a package from Symfony, you place trust in the maintainers of this package. For a package from Symfony, that may be easy to do. But does this apply to all the maintainers of all the packages you use? At least for making sure your dependencies don’t contain known security vulnerabilities there are tools available, like SensioLabs Security Checker and GitHub Security Alerts. But what about packages where it’s not that easy to put trust into them? Theoretically, on each update you’d have to vet the new version to ensure it’s still ok to use for you.

Packages vary in size – for small packages of just a couple files it may be worth considering not to add the package as a dependency, but to simply copy the required code you just vetted (with respect to the license, of course!) to your application and not bother with thinking about the dependency. This might be easier said then done, as this has a tradeoff: you lose simple access to bugfixes and more importantly security fixes. However, the component might not be very susceptible to security flaws. The additional burden of maintaining this piece of code yourself might be worth it in constrast to the burden of caring about the dependency throughout the future.

On the other hand, adding it as a dependency will provide you with easy access to bugfixes and security fixes. But are you sure your dependencies are really updated often enough? Too often I’ve seen cases where developers don’t update and applications fall behind. Just because it might be easy to update it’s not done automatically. This might have reasons like missing unit tests so you can’t be sure that an update doesn’t break anything, even if it’s supposed to be a bugfix update only. To ensure upgrades happen often the existence of tests is necessary, but not sufficient. I believe dependency updates must be automated, and luckily we see the first steps in that direction, like automated security updates from GitHub. This is great, but I think we have the necessary ingredients in place to automate all bugfix- and probably minor updates this way, and we definitely need this as the amount of software just grows bigger and bigger.

As always in software, there isn’t a definitive answer, and decisions taken vary from situation to situation. But being aware about what adding an additional dependency (or more than one if this one itself has additional dependencies) entails is definitely not a bad thing.

Lessons Learned from Testing and Refactoring Legacy

I remember when I first discovered automated testing. I immediately wanted to apply it to all the projects that I was working on, but it didn’t work as well as I expected. In fact, it was a disaster, which is why so many developers shy away from tests after a few failed attempts. It turns out that adding tests to a project that never had any tests is a much bigger challenge than testing new software. People get thrown at the deep end of the pool, then either learn to swim or get scarred for life.

The problem with legacy applications is that they almost never follow SOLID or clean code principles. This makes them hard to unit-test. You’d need to refactor the code before you can test it, but how do you know that the refactoring won’t break existing functionality? You’d need tests to ensure that the refactoring goes smoothly. We are in a deadlock… or are we?

Characterization Tests

When I want to refactor code, I don’t start with unit tests. I start with characterization tests, which are meant to characterize the current software’s behavior. I usually write them by calling an HTTP endpoint or batch script, then inspecting the output.

For example, I want to make sure that product prices are correctly synchronized based on a daily CSV file that we fetch from an FTP server. I will write a test for that and when I refactor the underlying code, this test would still pass, because it does not concern itself with the internal working of the code. It only cares that the end result is the same. Characterization tests will survive a refactoring operation.

Note that these tests are meant to ensure that the existing behavior doesn’t change, not that the behavior is correct. This means that if I uncover bugs, I’ll document them in tickets and continue testing.

Later, when refactoring is done, I can update these tests to capture the correct behavior that is expected, then fix the bugs that pop up, along with any unit tests that I have written during the refactoring. They will then become acceptance tests.

Exploratory Testing

Once I built myself a safety net with characterization tests, I need to understand how the existing code works. For this, I use exploratory testing. These are often throw-away tests that I only use to understand the code, its design and where things can be refactored.

I don’t refactor everything at once, as that would be a 2-year effort on some projects. I try to find smaller components and make them testable, but not so small that the refactoring will be pointless. With some practice, I eventually found the sweet spot.

To decide how to refactor, I look at difficult tests. The following testing points usually indicate an underlying code design flaw.

Hard Dependencies

These are easy to spot. When my test makes me mock a static call or a new keyword, I know that I will need to move these things to constructor dependencies.

I will also extract an interface from these dependencies, so that my code would respect the “D” in SOLID: one should depend upon abstractions, not concretions. Dependency inversion is the first thing that I will introduce into any legacy to make my tests sane and any code changes a lot less painful.

Uncontrolled Variables

Sometimes, my test will depend on things that I cannot control, like the current time or a user’s IP address. This is a great opportunity to refactor the code to not explicitly depend on these things.

  1. I will find all places that use mktime(), new DateTime(), etc.
  2. I will create and inject a Clock interface as a dependency for these classes.
  3. I will replace the time creation with $this->clock->now().
  4. I will write an implementation for it using DateTimeImmutable and another using a value that comes from a database, which I can later leverage for controlling the time in acceptance tests. The acceptance test writes a date to the DB and the application under test reads it.

Not only do these interfaces allow for robust and repeatable unit tests, but I can also simulate things like cache expiry or other interval-based logic.

Mixed Concerns

Sometimes, a class requires me to mock 15 dependencies, but only a tiny fraction of them is used in each method. This often happens in the context of an MVC framework where it’s common to have one controller class with many actions.

$controller = new ProductController($dep1, $dep2, ...);

Perhaps this class is mixing too many concerns and should be split into smaller classes. Group the classes by the dependencies that they have, even if it means one method per class. Instead of having a class called ProductController, you’ll have ProductListHandler, ProductViewHandler, etc. The resulting classes will be much easier to test, and the code will be easier to debug and modify.

Long Methods

Is a method too long and requires 200 unit tests, each with a ridiculous setup? Myself from 10 years ago would have written several thousand lines of unit tests and then hoped to never have to understand them again.

Today, I will split that into small private methods, group them by dependencies and move whatever makes sense into separate classes.

Let’s say that each comment in this example corresponds to about 20 lines of code:

public function synchronizePrices(): void
{
    // load CSV from file
    // parse CSV
    // create product array
    // if product doesn't exist, throw exception
    // look up product in database
    // if price is different, update it
}

As a first step, I’ll extract the code into private methods and ensure that my characterization tests still pass:

public function synchronizePrices(): void
{
    // This depends on the filesystem.
    $csv = $this->loadCsvFromFile();
    $parsedCsv = $this->parseCsv($csv);
    $products = $this->getProductsFromArray($parsedCsv);

    foreach ($products as $product) {
        // These two depends on the database.
        $this->findProduct($product);
        $this->updatePrice($product);
    }
}

Now I should move the code from those methods into new classes that I’ll inject through the constructor. More than 100 lines of code turn into this:

public function __construct(
    ProductRepository $sourceProductRepository,
    ProductRepository $targetProductRepository
) {
    $this->sourceProductRepository = $sourceProductRepository;
    $this->targetProductRepository = $targetProductRepository;
}

public function synchronizePrices(): void
{
    $products = $this->sourceProductRepository->getAll();

    foreach ($products as $product) {
        $this->targetProductRepository->updatePrice($product);
    }
}

Notice that both our repositories are implementations of the ProductRepository interface. Basically, I want to be able to synchronize between two repositories. This class doesn’t need to care how things are stored. I’ll just instantiate it with the CSV implementation on one side and the DB implementation on the other.

Maybe the third party will one day stop uploading a CSV and instead I’ll need to fetch the prices from a REST API, which I know just requires an additional implementation that can be done in an afternoon or two.

Of course, nothing prevents me from splitting things further inside of those concrete classes if they are still too complex. I will end up with smaller, testable classes.

More Refactoring

This is by no means an exhaustive list of all the refactoring that can be done. For more ideas, please read “Clean Code” by Robert C. Martin, “Working Effectively with Legacy Code” by Michael Feathers and “Modernizing Legacy Applications in PHP” by Paul M. Jones.

However, don’t try to do everything at once. As soon as you have a small class that depends on just a handful of interfaces and a rather short method, like in the synchronizePrices example, go write unit tests.

Unit Tests

Refactoring will make the synchronizePrices method much easier to test. Here is what the exploratory test might have looked like:

protected function setUp(): void
{
    // 50 lines to mock hard dependencies
    $this->synchronizer = new Synchronizer();
}

public function testSynchronizePrices(): void
{
    // 50 lines to build CSV content
    // create the CSV file and upload to FTP
    // 200 lines of mocks for the ORM calls
}

Here is how I would generally write the test after the refactoring:

protected function setUp(): void
{
    $this->csvProductRepository = $this->createStub('ProductRepository');
    $this->databaseProductRepository = $this->createMock('ProductRepository');

    $this->synchronizer = new Synchronizer(
        $this->csvProductRepository,
        $this->databaseProductRepository
    );
}

public function testSynchronizePrices_WithProducts_WillUpdatePrices(): void
{
    $this->csvProductRepository
         ->method('getAll')
         ->willReturn([$product1, $product2]);

    $this->databaseProductRepository
         ->expects($this->exactly(2))
         ->method('updatePrice')
         ->withConsecutive([
             [$product1],
             [$product2],
         ]);

    $this->synchronizer->synchronize();
}

In the unit test, I instantiate the class with a stub and a mock, then make sure to cover the two execution paths (with and without products). If I expect updatePrice to throw an exception, I can decide to add yet another test case and then implement a try/catch in the code.

As you gain a better understanding of what constitutes good code design, your tests will become increasingly easier to write.

Helpful Tools

In addition to all these tests, I also rely heavily on PHPStorm, which offers automated refactoring tools and a plethora of inspections that highlight any potential error that I’m making, like trying to call a method on a property that is possibly null: $entity->relationship->getName().

I go even further with PHPCS, PHPStan and Psalm, all at once, to break my CI in case I write something that has even a remote possibility of being incorrect. Because legacy code lights up my CI like a Christmas tree, these tools can be configured to only apply on files that you have edited.

I also had some great experience with RectorPHP, which can automatically improve things like moving static calls to constructor dependencies, add type declarations, etc. There are many great tools in there to give you a refactoring boost.

If you have a problem, there is a good chance that someone solved it years ago, wrote a book and maybe even created a tool for it. Keep exploring and have fun!