Pipelet

The lack for concurrent access proves to be a real issue when the number of workers increase as it prevents completely access from the web when running.

In addition, it occurs that computing infrastructure do not provide any filesystem able to hold a database. As an exemple, CCIN2P3 filesystems accessible to users are all distant filesystems with significant latency. This induces a serious performance hit that prevents in practice the number of tasks to exceed a million. It would be interesting in such cases to have dabases hold by a distant and better suited server.

the mysql python backend support DB-API 2 as the sqlite3 backend so I guess it is not that a big work to write a new tracker backend for mysql.

mysql-python.sourceforge.net/