Loading...

MySQL + SSL + Doctrine

Post thumbnail

Enabling and enforcing SSL connection on MySQL is easy: Just generate the certificates and configure the server to require secure transport. And it’s even easier on the Azure Database for MySQL.

Just go to the Azure portal and enable “Enforce SSL connection” in the “Connection security” of the database server. Then, download the certificates and stuff them in your application. No problem — just two simple steps.

On a hackathon, we’d just download the certificate, put it in a random folder, and move on. In reality, we’re in a bit different situation then. We’d have three (so far) services connecting to the server...

Automating Self Hosted Pipelines — Part 2

Post thumbnail

In the first part of the article, I described how I automate the worker virtual machine setup using a Bash script. Now I’ll describe how to automate that Bash script using Azure Pipelines.

Having gone through the first part of the journey, I now have a Bash script which takes two environment variables — a token and a worker name — and sets up a new Azure pipeline worker. That’s cool but I still have to create the token, write the instructions how to run the script, store the script somewhere, run the script somewhere, etc. Still some room for automation and...

Automating Self Hosted Pipelines — Part 1

Post thumbnail

As part of our movement away from Travis CI to Azure Pipelines, I came across a repository which needs to run docker commands with swap limiting capabilities. It is included in our application which runs docker containers and is in fact an important feature enabling the application developer to specify some of the container limits. When testing, we’re also testing the memory limiting some docker containers, which understandably does not really work on Microsoft-Hosted Azure Pipelines agents. It does not allow such low level operations:

This is a two-part article. First, I’ll describe how to automate the worker...

Kubernetes Pre-Stop Hook Reflections

Post thumbnail

I have an application which runs as a Kubernetes job. The application is implemented as a PHP Symfony console command, which is specified in the Dockerfile entrypoint:

When the image is built and the container starts, the PHP process runs a console command app:run. When the command terminates, the container terminates and the Kubernetes job finishes. So far, nothing special.

The next thing on my list was to implement termination of the job. This is actually trivial to do through kubectl (or the corresponding API):

That’s all. However, here comes the interesting part. The app:run command process spawns some...

Flee from Travis

Post thumbnail

Travis CI became a huge PITA last year. Basically, it randomly fails for hours or sometimes days. There are no status updates, no communication from support, etc. This is for another post.

Yesterday this hit me on a hugely legacy app, which has a pipeline split in 7 stages of building some docker images and executing over 350 functional tests. It’s a beast which takes slightly over an hour to build and unfortunately relies on Travis build cache which broke yesterday:

Looking at the pipeline — it is actually very simple, since most of the stuff is wrapped in test suites. So instead...

Pagination testing with Guzzle Mocks

Post thumbnail

Having a lot of microservices means that writing a service client become a daily routine for me.

Writing a client for a REST API service is usually nothing too difficult. I strongly favor this over just ad-hoc calling some endpoints via Curl or Guzzle. The client can handle all error handling and retry logic and possibly other things (like API throttling). And mainly, it is fully tested. I usually write a few functional tests and then use mock tests to achieve 100% coverage. While there are many discussions about the benefits of 100% test coverage (I find this article)....

PHPStan with Doctrine

Post thumbnail

Recently PHPStorm 2020.3 was released with first class PHPStan support. Eager to try it I enabled it for a current project. The project is a bit of a pickle because it was put on hold a year ago. When resumed we had to start with rewriting half of the code and major refactoring. In the meantime I also boldly updated all the dependencies irrespective of any BC breaks.

This is how it looked:

212 errors — pretty bad for an application which was passing the max PHPStan level a year ago.

We’re using PHPStan from within composer.json like so:

This is a useful concept, because...

Running Composer update on Ancient code

Post thumbnail

PHP 8.0 was recently released. At the same time, I’m still occasionally working in an archeological dig on a hugely old PHP application. It’s so hugely old that it runs on PHP 5.6 only and depends on a framework which depends on Symfony 2.8 (that’s why it runs on PHP 5.6 only).

On an unfortunate occasion, I also need to use composer to run composer install on this relic and it fails miserably because of exhausted memory. The solution is to run composer update to bring the dependencies forward (there is more than 120 of them) which of course fails...

Testing the untestable

Post thumbnail

An important part of Keboola Connection are Extractors — especially Database Extractors. These components load data from different (SQL) databases and import them into Keboola Connection. They are in principle very simple — they execute an SQL query and export the result into a CSV file. As with all simple things, the devil is in the detail. One of those details is retrying on transient errors. Usually there is a substantial amount of data returned by the SQL query, so the export easily takes few hours and we don’t want short network interruptions to break it.

This is where the devil comes into the...

The Downfall of Exception

Post thumbnail

Some time ago I came across a weird situation. I had a piece of PHP code which behaved really strange. Since the actual code was a rather complicated beast, I’ll use a simplified example. Let’s have three functions:

So — I have a function a, which somewhere in its code uses a PHP function throwing a warning or notice — e.g. fopen. I check for the return value and if that function fails, I raise an exception. The outer function b catches the exception and retries the call if needed. So far, so good, ain't it? I also have a PHPUnit test for it:...

Profile image

About me

I’m a senior software engineer currently working at Keboola.

I’m currently mostly a backend developer, but in 15+ years experience of software development, I’ve done a lot of different things. See LinkedIn for the official biography.

Nowadays, I’m mostly working mostly with PHP and Python. I design and implement scalable, maintainable and testable APIs. My daily work ranges from setting up the infrastructure via Terraform and setting up the CI pipelines, via writing the code and tests to designing interfaces and entire system architecture.

I’m a big fan of automation and testing. As such, our team of 4 engineers is able to work and maintain 2 mono-repositories and 95 repositories, which altogether contain 16 domain services, about 100 shared libraries, all totalling to more than 500k lines of code in different languages. Yeah, so automation and testing is a must.

Some superlatives I encountered during my Journey:

  • The language in which I wrote the least amount of code that still made sense – 8086 Assembler – I created one or two levels of a Sokoban game in just a few hundred bytes of code, but unfortunately, I lost it long ago.
  • The language that brings back the best memories – C# – “You fell asleep with your head on the keyboard, and you wake up to find the app is written.” It really felt like that, but then I moved to web development.
  • The language in which I wrote the longest function name – C++ – It was about 300 characters long and ended with an evacuation of the building. Don’t ask for details.
  • The language in which I completed my first paid work – Delphi – It was my first real job, and they let me choose the tools since the licensing was permissive at the time.
  • The language I’ve had to relearn the most times – Javascript – It started off pure, then came JQuery, followed by Node.js, ECMAScript 6, and now TypeScript…
  • The language I abandoned the quickest – Julia – Once the task was completed, I simply had no further use for it. It’s nothing personal.
  • The language in which I’ve written the least code overall – Lisp – I only wrote one function.
  • The language in which I’ve written the most code – PHP – By an order of magnitude.
  • The most ambivalent language – Python – While the language itself is impressive, there’s an overwhelming amount of poorly written code in it.
  • The most exasperating language – R – Have a vector with one number and ten nulls? It will return the number as an integer. Were you expecting a vector? Too bad.
  • The first language I ever coded in – Sinclair Basic – I was too young to remember what it was, and thankfully, it’s long gone (my code, not Sinclair).
  • The most eccentric language – Squirrel – It’s akin to Lua, but bears a closer resemblance to PHP, except for certain aspects that are more Python-like.
  • The language I was happiest to forget – Visual Basic for Applications – Until I had to learn it again… and again… and again…