Ticketea.com is the platform where we showcase and sell the majority of our events. The software project powering it (code-named "Aphrodite"), is a moderately large application originally built using Python 2.7, and it was kept updated when new Django versions came along. It currently runs on Django 1.11.
Python 3 is the present of the Python language, and all of our new projects are being written using Python 3.6. When adding new features to ticketea.com, we often came across compatibility issues with components written for newer versions of the language. We decided not to delay the Python 2.7 → 3.6 migration anymore, and started working on it.
All of our projects come with a Dockerfile, so the first step was to change Aphrodite's base image to be
This triggered the first issues:
- Outdated external libraries which needed to be updated to newer versions featuring Python 3 compatibility
urllib.urlparseand similar major changes
- Dictionary change like
.items()now returning a view.
- Things that weren't needed anymore, like Django's
Many of these issues were corrected just by running 2to3, which not only fixed many of the compatibility issues, but also applied patterns such as rewriting
lambda to list comprehensions. Way safer than doing raw
Once we finished working on the "low-hanging fruits", the next step was to run Aphrodite's test suite and achieve zero errors. We have multiple satellite Python libraries and components, and while most already had a Python 3 compatibility branch, a few were outdated and missing the latest bugfixes coming from master. Once this was fixed, we discovered a few bugs while performing integration tests. We ended up fixing things, increasing the test coverage of those components, and those Python 3 branches won't be forgotten again (and will be merged into master).
Along the way, we applied the boy scout rule of "Always leaving the campground cleaner than you found it". So we fixed a few Django 1.11 deprecation warnings to prepare for Django 2.0. As it usually happens with Django, the official documentation was excellent, explaining very clearly how to implement the necessary changes.
Things we learned during testing
Code coverage was originally around 70%, so of course when something failed, it was always coming from that remaining 30%. There were no major issues and we have more tests as a result, but like always, the more tests you have the better.
Keeping the Python 3 branch up to date with
master: This is a considerable effort unless you're able to freeze any change to master (very unlikely in the real world).
A non-trivial feature was delivered during the migration. Due to the amount of modified code, we thought it was easier to merge the feature branch into the Python 3 branch ASAP, and keep them synchronized.
pickle protocol version in python 3 can be higher than the highest available in Python 2.7. So we needed to add versioning to our Django caches, as python 2 goes only up to pickle v.2 (caches with python 3.6 serialize with pickle v.4).
Each modified file had to comply with flake8 linting rules. We recently setup a "linter test" that grabs all files you are modifying on the current branch and runs
flake8 against them. This meant that, as we had to modify around 200 files in total, all of them had to now be clean and perfectly formatted python code :) We left out Django migrations with long string literals, and fixed the rest. It took some extra effort, but the results are worth it.
When the test were all green, we started working on the harder part: Manual integration tests, first in a local dev environment, then in our staging environment.
Going to production
After staging tests were all fine and error logs didn't registered any issue, we proceeded to deploy to production.
Afrodita is currently running on Google App Engine Flexible, and one of the features our team loves with is traffic splitting:
With this feature, we can do canary releases with ease: We just deploy our new version of the service, and start redirecting small amounts of traffic traffic while we monitor for unexpected errors.
After some minor bugfixes, we could bring the traffic of the Python 3.6 version to 100% with confidence. We also had the old version available for instant rollback, thanks to how parallel versions and traffic splitting work in GAE flexible.
We finally stopped the Python 2.7 version (which was receiving zero traffic) and merged the Python 3 branch into
And that is all. As the final note, this migration was handled by just one developer and our QA engineer. When you have the right tools you can minimize overhead and focus on what really matters.