Site news

We've launched the Drupal.org redesign theme on localize.drupal.org about six weeks ago, and the reception was great. While other subsites like api.drupal.org are also in the process to migrate to this theme, we could pioneer some fixes and get them into production. We keep tweaking the theme on this site and get fixes in based on your feedback.

Some great feature additions landed since the last update. The most requested new feature is that you can now export all outstanding suggestions with translations. In case of multiple outstanding suggestions for any one string, the suggestions will be in comments. In case of single suggestions, the export uses Gettext's fuzzy facility and just marks the string as "not ready". Look for this option on the language export screen.

Another great feature which should help translation sprints and discussion around concrete string translations in general is the possibility to link to each individual source string. You can reveal the list of links for each string on a translation page by clicking on the # (hashmark) in the source string table header. This will show links with IDs for each source string. You can bookmark these or use them to discuss strings in forum discussions or chats. The "Welcome to Drupal..." string in Hungarian for example can be deeplinked as http://localize.drupal.org/translate/languages/hu/translate?sid=251536

We also put down our legs on drupal.org! Now every project page on drupal.org includes a link to "View project translations". For example http://drupal.org/project/views will link to the corresponding project page at http://localize.drupal.org/translate/projects/views which gives you status information and download links to specific Views translations.

Since the last update, we've opened some new teams. Recent additions include the Portuguese International, the Gujarati and the Haitian Creole team. Welcome! The process for new team additions is explained on the front page.

Finally, on some technical notes, the main drupal.org site was the first to employ a Bazaar and Hudson based deployment system recently to ease our collaboration on new features. This week localize.drupal.org joined this system, so our code is now fully hosted and managed there.

Mark Boulton Design was selected to lead the drupal.org redesign efforts almost two years. While they went on dutifully with the process listening to feedback from the community and designing a modern home for all things Drupal(.org), and the deliverables were in on time, the actual implementation (which was on the drupal.org community in general) lagged behind a bit. I've personally been to multiple sprints where we established the base for the redesign infrastructure on many levels. The big problems are not around applying a different theme to the drupal.org site but reworking the structure and some sections for the redesign. The latest sprint which I was unfortunately unable to attend was in San Francisco, where in a surprising turn of events, it was set that drupal.org subsites should push forward with implementing the new theme and concepts even ahead of drupal.org. After all, we do not have big rearchitecture plans for these sites, so we can sidestep the biggest tasks which keep drupal.org behind. So came http://association.drupal.org/ up in the new theme shortly after. This got me inspired to get deep down into applying the theme to http://localize.drupal.org/ as well.

To see how far we are, I've put the new theme under localize.drupal.org for a test drive a few weeks ago, and it was not looking nice. Turned out that some developments pushed it away from the underlying navigation-sharing infrastructure we built earlier. So I worked on getting those advantages back and making all the shell of the theme work. Then came putting in the localization server pieces to the right places. This mostly involved making some obscure custom page elements actual blocks to put them into the right place with the new theme. I believe this helped push both the theme and the underlying navigation-sharing goodness to get into a better shape for use on other subsites in the hopefully not too far future.

Finally today was the day to finish it all up, fixing some odd issues and switch over to the new theme for your enjoyment. With this, we join the Drupal Association site in adapting the new theme and hope to provide you with an increasingly integrated experience as other subsites adopt the same theme and shared navigation structure.

Drupal.org redesign, here we come!

(Note that anonymous visitors might get half-baked cached pages while the caches are refilled with all the new pages. Let us know if/when you find issues with the new theme via the usual means.)

Just last week, I've decided to finally document the current download methods for translations coming from localize.drupal.org at http://localize.drupal.org/download. We've set up translation packaging earlier this year and let it run while being monitored for tweaking. We found that we can run the packaging as often as every 5 minutes and with allowance to generate as many as a hundred .po files per run. With these numbers in mind, we can generate 100*12 = 1200 files per hour. But we have 67 languages and 12000 releases to generate .po files for, so that is above 800 thousand files to generate. If we'd need to regenerate all files all the time, it would take 28 days.

Looking at the database, we've generated close to 500 thousand files by now and the drush processes looking at generating more found that its time to update the existing ones instead. We do not generate .po files for language and release combinations where no translation exists, so 500 thousands versus 800 thousands is a good amount of files generated. Given we have all these base files now, all our drush jobs do now is they regenerate files as needed. With them being set to look at 100 releases at once (and sometimes still not finding any translations to regenerate), we seem to be on to regenerate files as new translations come in pretty fast (think days or even hours). We are still experimenting with different parameters and ways to speed the process up, but it looks promising.

Let us know if you have any feedback on the downloadables and especially if you have good/bad experience with http://drupal.org/project/l10n_update, which I'd suggest you try out to automate the flow of getting translations for your sites.

Our last major update to localize.drupal.org was about two months ago, bringing in a whole new user interface for translation and easier navigation and collaboration. Since that was a big user interface change and needed some time to settle, I switched gears to work on the backend of the system. The ultimate goal is to get our main backend module called Localization server as general purpose as possible.

Many people noticed that the software on localize.drupal.org would be nice to use for translation of other projects so generalizing the software to get in more collaborators and contributors was a logical step. The 6.x-2.x-beta2 version of the module was released earlier this month with a blog post explaining its general use titled Using Drupal as a collaborative software translation tool. This new release included some pretty big architectural changes, rebasing the module on l10n_server and starting to separate the database backend, the Drupal specific connectors and the exporters more.

There is still quite some work to do around there, but it was time to make the jump and roll out this new backend on localize.drupal.org as well. This time the rollout went well if you do not notice any changes. While there were minimal UI fixes as well, generally nothing should have changed on the front end. However, the backend now provides more tools to administer the server, so in case of the parsing errors we occasionally notice for example, we can easily reset a release for parsing again on the user interface.

Let us know if you notice any new problems. See the having issues page for explanation on how and where to submit bug reports. Thanks!

As I've blogged about this before, work on localize.drupal.org's improved functionality was going on heavily in the past weeks. We launched a staging site with interface improvements almost a month ago, and kept adding small improvements on top of that.

Given all the great feedback on the new user interface, it was time to finally launch it on the live site. Here is a rundown of the most important improvements.

We are going to perform some exciting updates on localize.drupal.org tomorrow, on March 17th, 2010, from 8am to 10am UTC (expected). In this timeframe, all of localize.drupal.org services will be down and inaccessible. None of the other drupal.org sites are affected, the main site, groups, api, etc. will be fully functional. We are sorry for any inconvenience caused, and thank you for your understanding.

I've announced our staging site for localize.drupal.org two weeks ago, and some people did take on the opportunity to test out the site and provided valuable feedback. Fixes and improvements are rolled out continually on the site. Since the staging site was set up with a completely revamped user interface to translate text, the following changes made their way onto the staging site:

• [#225728]: the front page is now streamlined, gives more prominence to the highlighted project and stats instead of boring you with static text
• [#735218]: groups you are a member of are now prominently highlighted on the front page
• [#737230]: the appearance of the filter forms was highly simplified and made more discovery-based
• [#727574]: missing tooltip help for the red circles and blue arrows were added and existing tooltip and UI text was revised / simplified
• [#738174]: Organic Groups functionality was tightly built into the translation UI instead of relying on navigating away and getting lost
• [#738194]: Breadcrumbs were reviewed and fixed to ease navigation

While we did not encounter showstopper feedback in the past two weeks, I'd like to keep the new interface on the staging site for a bit more and look for more feedback before pushing it live. Feeling from all the feedback so far, I'm certain we could relaunch the live site with these changes before / in-time for Drupalcon San Francisco, so if any translation teams organize translation sprints for that time, they could enjoy all the new goodies.

Remember: all your data entered on the staging site will be lost, since the live site database will be used to run the updates. So please do use the new site as a testing ground, but not for actual serious work.

Test drive with us!

We suggest you read the details and tips here, but if you'd like to go ahead and try this out now, just go to http://localize.staging.drupal.org, enter drupal/drupal for the HTTP authentication prompt when requested, then log in with your drupal.org username and password and play around. Have fun and provide us feedback! Thanks for your help.

Looking back a bit

The main translation user interface of localize.drupal.org as we know it today is a result of collaboration between myself and the amazing Young Hahn of Development Seed. His work with the user interface covered making a previously unsexy interface more appealing and useful. However, given how early that was done, we did not know the actual user scenarios of language teams having contributors of all levels.

Looking at the issue queues, the most pressing problem on localize.drupal.org is the "uncontrollable" flow of suggestions. Of course you can call this the success of our web based translation interface. There is a huge number of suggestions coming in. If you look at the numbers, we have over 300.000 translations recorded and over 100.000 suggestions as well in the system. So 25% of the active text in our database is not yet approved or declined.

I've got various requests of both translators and moderators to the effect that the translation user interface is inadequate to handle the suggestions. The UI is very inviting to submit new suggestions, but the existing ones are hidden behind a little hardly visible star:

Additionally, if you click on the suggestion star, the original translation is hidden, so you cannot tell the difference between the two or more pieces of text. Clearly, both to handle existing suggestions and submit new ones, seeing the current translation with all the suggestions listed makes processing the community submissions much easier.

Drupal 7 Alpha 1 was just announced, while its tarball and release notes page was already available before. Because the package was up earlier today, localize.drupal.org's robot already went ahead and parsed the first Drupal 7 package source code. This means that now you are able to start work on the translation on the localize.drupal.org interface.

You can find out about the amount of new and changed strings (which are numerous) by going to your language team page (pick your language team and type Drupal on the front page of localize.drupal.org). Then on the translation page shown, pick the first 7.0 alpha from the release dropdown. With the landing of Drupal 7, you'll also notice the appearance of the context filter. Drupal 7 now supports contexts, so the translation of "May" can be different when in the "Long month name" context when it is translated without context (which assumed to be the short month name version). Watch for more use of contexts in the contributed module and theme space!

While Drupal 7's first alpha is intended to be generally stable in terms of UI text, translators are good in finding errors, omissions and outdated text. So do not consider this mass of text the absolute final version of Drupal 7's user interface.

If you went through previous major Drupal releases before, you might miss a crucial feature on localize.drupal.org, namely "fuzzy matching" of previous strings. When some text was only slightly changed, this fuzzy matching could offer the previous translation as a suggestion to help you start off with the translation. This is a great feature of several Gettext tools. Unfortunately we did not yet implement this due mostly to performance constraints. However, Hilde Austlid (aka zirvap) wrote up a nice summary for you on how you can do this with helper tools outside of localize.drupal.org to help you bootstrap your Drupal 7 translations. Merging in previous versions of the CCK, FileField, Imagefield, and other, now core-integrated modules is a genuinely useful tip. Many of the "new" strings in Drupal 7 are modifications or straight copies of text carried over from integrated contributed modules. While localize.drupal.org will pick up the straight copies, the slight modifications it will treat as completely different strings, so this msgmerge trick can be very useful.

Let us know if you find issues translating Drupal 7 (or any other problems on localize.drupal.org for that matter), our having issues page explains where to submit your problems or requests.

In the past several days, I've been working on several bugfixes and performance improvements, preparing the site for the imminent landing of Drupal 7's first alpha release, which will also be the first tarball version of Drupal 7 and mark the string freeze stage of development on this latest Drupal version. What that means, is you might start flooding the site exporting and importing .po files as well as generally scramble to translate Drupal 7 and beat other language teams to it. You'll surely notice the start flag in a couple days to kick off the race.

Localize.drupal.org has support for Drupal 7's localization API for quite some time now, so that should not be an issue for us. However, there were some (admittedly fairly badly coded) SQL queries used when new release source codes were parsed and your .po files were imported, which were holding our servers down unnecessarily. Following a quick tip from Drupal.org user droplet, I've looked into implementing more elegant string matching solutions, which resulted in amazing performance increases when importing .po files and parsing source code for new releases. Updated code was deployed earlier this week. This should largely help with the potential flood of .po files coming in the following weeks.

While having translations on localize.drupal.org is fine, getting it to your Drupal sites is even more interesting. Therefore Jose Reyero picked up development of this functionality and is doing it in a new Localization update module. The reason it is not part of an existing module is that this will be required in any install profile which would need to be localized and bundling in a whole set of unrelated functionality (like the whole Localization client) might not be desired. There is also a Localized Drupal install profile which is forming to be a base install profile for a localized core-only distribution. We are preparing tools for install profile builders to include in their profiles to language-enable them as easily as possible.

Finally, unfortunately there is not that big of an activity around reworking the translation UI to a fancy new user interface that many people said they love. It would be great if more people could help out there refactoring this huge patch to smaller digestible chunks, so we can get the improvements in. Working with, reviewing and testing a 116K patch is no small feat. Thanks to anybody who is helping out.