Greater than 3 minutes, my friend!
Developer Diary 22: On breaking stuff and being impatient
Oh my!
It’s been a veeeeeery long time since I published anything in my Developer Diary.
I mean, I do publish articles every now and then (for example, when I announce new exciting features or write recaps of #BlabbingTranslators), but I haven’t really shared much lately about all the work I do behind the scenes.
Mostly, well, because I do the work and don’t have much time for writing comprehensive reports about everything I do anyway. 🙂
But sometimes I do have the urge to share some thoughts with my developer hat on.
Most of a time it happens when there’s a teachable moment I’d like to share or some very interesting insights.
And in this case it’s the first one.
Yesterday we had a moment that have clearly demonstrated what it’s like to have a non-developer guy trying to fix things that ain’t broken.
Yep, you guessed it. That non-developer guys is me, Chief Master of Breaking Stuff reporting for duty.
We had a major service interruption yesterday that lasted for about 4 hours all thanks to my “brilliant” idea of updating a few things.
New version of WordPress?
My, oh my! It’s very tempting to hit that update button!
So I did and as a result I broke The Open Mic and almost put the entire galaxy at risk.
This GIF pretty much sums up my relationship with WordPress
And here’s the interesting bit: there was no point in hitting that update button, really.
Once again, if ain’t broken why fix it?
And the reasoning was pretty simple: i just wanted to try and see what happens.
Yep, I was feeling lucky that day.
Unfortunately, web-development is rarely about luck.
And it’s rarely about building cool awesome things and getting your users excited and fired up for things to come.
9 times out of 10 (or 99 times out of 100 if you prefer a 100 points scale system) it’s about grueling, often boring, and most certainly never-ending bug hunt and mundane, repetitive, non-sexy tasks (like testing, ensuring compatibility between different pieces of software and troubleshooting).
It doesn’t sound exciting, does it?
But that’s what I do almost every single day.
I build stuff, I test stuff, stuff gets broken, I think I fixed it, then it gets broken again, then I fix it again, then it gets broken again or even worse – a user finds a way how to break it (they always do, it’s like they have a super power) and then I need to kick ass at support and customer service to make everyone happy and, of course, fix stuff, which, eventually, will get broken somewhere down the line.
It’s a never ending circle, but hey! This is what you get when you have non-developer guy, well, developing things.
That’s why yesterday I was feeling a bit lucky and impatient and jumped the gun a bit too early without testing stuff first (FYI: we do have a staging site for all of my crazy experiments, I was just being lazy lately and didn’t keep those 2 in sync by pushing my updates straight to the live version without any tests on the staging website first).
I was lucky a couple of times, but yesterday it wasn’t the case.
The Open Mic is down at the moment due to maintenance. It might take around 30 min before it goes back up. Sorry about that.
— The Open Mic (@OpenMicXL8) May 4, 2016
But hey! At least I had this teachable moment about the importance of patience and diligence in web-development.
I hope you’ll be able to learn from my lesson too! 🙂
P.S.: Huge thank you and a bear hug to all the awesome peeps who bear with me while I’m trying navigate through this amazing world of web-development! We have some good times, we have some bad times, but I’m 100% certain that you can’t build something awesome without breaking a thing or two, or three, or a couple hundred thousands of things.
Thank you for all your continued support and encouragement and especially for flagging all those nasty little bugs and annoyances that you encounter on The Open Mic. That makes you as much of builders and developers of this community as me and this what makes The Open Mic so awesome and unique! 🙂
The Open Mic Team 
I’m also always afraid to press that button, I feel like I’m doing to denonate the atomic bomb again! Very frustrating. Now that you shared this I’m afraid to do it this time!
So you know that feel too, Patricia! 🙂 In my case it is probably much worse on the account that we have 50 plugins installed and one tiny update to any of them can break something in one of other 49, so it’s always a game of figuring out what will break what. But I’m getting better and better at it 🙂
Dmitry, for a long time I did not update my server software when I was asked to. But I learned an important lesson: if you postpone it all the time because “it ain’t broken, so why fix it?”, you end up with a system that is easy to hack. So, I update my system almost every time I get a reminder. What frustrates me is that the CMS systems that we all use, don’t predict trouble. That they don’t have a sandbox embedded, that would allow us to see some things won’t work anymore. But even a sandbox would not be a solution because it is hard to check everything, especially if you use 50 plugins.
Postponing an update can be bad for business as well. So don’t blame yourself for the problems caused by the update. These things happen, and will continue to happen until a great development team discovers a way to deal with it.
Thank you, Gert! I agree that we need to keep up with updates even if they break a few things here and there. That’s why I now have a staging site for The Open Mic where I test all the updates before pushing them to the live version. It’s not ideal because one man cannot test everything in such a complex projects as The Open Mic, but at least I can minimize the risks of breaking the website entirely when click that Update button.