Thanks to many hours of work from Jakub Suchy and myself, I've integrated and deployed many performance optimization and caching improvements on localize.drupal.org tonight. The front page and the main explore pages (explore languages and explore projects) are now using fully cached data, so will load way faster then before. This is on the expense of absolute accuracy. The cache is cleared and regenerated every half an hour, so when you add a translation or suggestion, it will not be immediately reflected in these stats. The underlying database also got some denormalization resulting in much slimmer queries, which will hopefully even be easier to profile and optimize.

The reason I'm so concerned about performance here is that you all move here to have a nice working environment, but adding more teams and especially entering more of those fine translations, the queries used to build the overview, listing and editing pages perform worse. We still have a long way to go since experiments on data sets with fewer languages but way more complete translation statuses showed that the current database / codebase performs badly and is still in need of major performance improvements.

Thankfully we have all latest drupal.org project releases parsed (data which can be used to build a performance testing kit), and the l10n_server now includes devel_generate style data generators for performance testing (with expectation to be coupled with a data dump from localize.drupal.org). This helps to quickly build up a site where real life performance of the current codebase can be looked at once the translations advance way ahead into where they are today. Similar performance problems arise obviously with more languages added even if relatively less progress is made in each of them individually.

We still have time to fix these performance problems until a bigger flood of translations and suggestions arrive on the site, but time is pressing. I'd love to see people helping out in the localization server issue queue to fix performance issues. I'm happy to collaborate with people who'd like to set themselves up with the performance testing kit to make this easier. You can also contact me personally.