Actions
Task #10294
open
Deploy treqs on test instance (ccsvli10) for test by PEB
Start date:
07/03/2015
Due date:
% Done:
50%
Estimated time:
Updated by Brinette Pierre-Emmanuel over 9 years ago
- % Done changed from 0 to 50
Déploiement de la dernière version disponible sur le repository :¶
http://maven.in2p3.fr/fr/in2p3/cc/storage/treqs/treqs-delivery/1.0-SNAPSHOT/treqs-delivery-1.0-20150703.143750-35.rpm
Arret de treqs :¶
# /etc/init.d/treqsd stop Stopping treqs-delivery... Stopped treqs-delivery.
Sauvegarde le version précédente :¶
# cp -rp treqs-1.7.0 treqs-20150626
Installation du package :¶
# rpm -Uvh --force treqs-delivery-1.0-20150703.143750-35.rpm Preparing... ########################################### [100%] 1:treqs warning: /opt/treqs-1.7.0/etc/logback.xml created as /opt/treqs-1.7.0/etc/logback.xml.rpmnew ########################################### [100%]
Configuration¶
Modification du fichier de log du wrapper¶
# diff wrapper.conf.orig wrapper.conf 64c64 < wrapper.logfile=logs/wrapper.log --- > wrapper.logfile=/var/log/jtreqs/wrapper.log 121c121,122
Fichiers de logs :¶
Activation du logfile et du tracefile dans logback.xml;
# diff logback.xml logback.xml.rpmnew 58a59 > <!-- 76a78 > --> 94c96 < <appender-ref ref="LogFile" /> --- > <!-- <appender-ref ref="LogFile" /> -->
Configuration de la BDD :¶
Installation des triggers et des procédures stockées :
# mysql -uroot -pxxxxxx treqsjobs < DBManagement.sql # mysql -uroot -pxxxxxx treqsjobs < archive.sql `
Démarrage de treqs¶
/etc/init.d/treqsd start Starting treqs-delivery... wrapper | Working directory set to: .. wrapper | Spawning intermediate process... ccsvli10_root# ls -ltr | tail -3 -rw-r--r-- 1 root root 4580 Jul 6 11:39 jtreqs.log -rw-r--r-- 1 root root 218327 Jul 6 11:39 jtreqs-trace.log -rw-r--r-- 1 root root 29813 Jul 6 11:39 wrapper.log
test¶
Depuis une ccdevli :
$ export TreqsDBHost=ccsvli10
$ /usr/local/products/treqsc/bin/trcp.pl -d /hpss/in2p3.fr/home/p/pbrinett/GOTCHA
2015/07/06 11:49:11 [15461] INFO> Treqs Client v0.2
2015/07/06 11:49:11 [15461] DEBUG> File to stage: /hpss/in2p3.fr/home/p/pbrinett/GOTCHA
2015/07/06 11:49:11 [15461] DEBUG> Resolved IP for hostname: ccosvms0007 -> 134.158.239.7
2015/07/06 11:49:11 [15461] DEBUG> Client IP adress is: 134.158.239.7
2015/07/06 11:49:11 [15461] DEBUG> Username is: pbrinett
2015/07/06 11:49:11 [15461] INFO> Connect to TREQS SERVER (ccsvli10) on DB (treqsjobs) as treqsc
2015/07/06 11:49:11 [15461] INFO> Timout set to (21600 s)
2015/07/06 11:49:11 [15461] INFO> First pool interval is 30s , Pool interval is 60
2015/07/06 11:49:11 [15461] DEBUG> Connection to Database (DBI:mysql:database=treqsjobs;host=ccsvli10) succeed
2015/07/06 11:49:11 [15461] DEBUG> Resolved IP for hostname: ccosvms0007 -> 134.158.239.7
2015/07/06 11:49:11 [15461] DEBUG> Insert file request:
INSERT INTO treqsjobs.jrequests(file,creation_time,user,client,email,version,status)
VALUES
('/hpss/in2p3.fr/home/p/pbrinett/GOTCHA' , NOW(), 'pbrinett', '134.158.239.7', 'pbrinett', 'treqs perl v0.2', 100) ;
2015/07/06 11:49:11 [15461] INFO> Request for file /hpss/in2p3.fr/home/p/pbrinett/GOTCHA succefully inserted, request id is (87867)
2015/07/06 11:49:11 [15461] DEBUG> Database disconnected
2015/07/06 11:49:11 [15461] DEBUG> Sleep 30 s before first pool
2015/07/06 11:49:41 [15461] DEBUG> Connection to Database (DBI:mysql:database=treqsjobs;host=ccsvli10) succeed
2015/07/06 11:49:41 [15461] DEBUG> Pool file request: select * from treqsjobs.jrequests where id = 87867
2015/07/06 11:49:41 [15461] INFO> Request id (87867) status : 150 (ON_DISK)
2015/07/06 11:49:41 [15461] DEBUG> Database disconnected
2015/07/06 11:49:41 [15461] DEBUG> Loop ended, status is : 150 (ON_DISK) Timout exceed : 0
2015/07/06 11:49:41 [15461] INFO> File /hpss/in2p3.fr/home/p/pbrinett/GOTCHA allready on disk
2015/07/06 11:49:41 [15461] INFO> Program terminating: return code is (0)...
/var/log/jtreqs/jtreqs-trace.log¶
2015-07-06 11:49:36,411 [Dispatcher] INFO Dispatcher - Looking for new requests
2015-07-06 11:49:36,412 [Dispatcher] INFO MySQLBroker - Mysql access duration (MySQLBroker.executeSelect method) took 0 ms (0 s)
2015-07-06 11:49:36,415 [Dispatcher] INFO Dispatcher - Beginning MetaData fishing on HSM for 1 files
2015-07-06 11:49:36,423 [Dispatcher_req_87867] INFO AbstractHSMBridge - Testing keytab: /var/hpss/etc/keytab.root
2015-07-06 11:49:36,442 [Observer] INFO MySQLBroker - Mysql access duration (MySQLBroker.executeSelect method) took 1 ms (0 s)
2015-07-06 11:49:36,571 [Dispatcher_req_87867] INFO HPSSJNIBridge - Hpss query attributes duration for file /hpss/in2p3.fr/home/p/pbrinett/GOTCHA was 136 ms (0 s)
2015-07-06 11:49:36,571 [Dispatcher_req_87867] INFO HPSSJNIBridge - filename '/hpss/in2p3.fr/home/p/pbrinett/GOTCHA' '{ position: 0, offset: 0 }' storageName 'DISK' size '9'
2015-07-06 11:49:36,572 [Dispatcher_req_87867] INFO Dispatcher - File /hpss/in2p3.fr/home/p/pbrinett/GOTCHA is on disk, set the request as done
2015-07-06 11:49:36,579 [Dispatcher_req_87867] INFO MySQLReadingDAO - Mysql access duration (MySQLReadingDAO.setRequestStatusById method) took 4 ms (0 s)
2015-07-06 11:49:36,580 [Dispatcher_req_87867] INFO Dispatcher - Dispatching duration (innerProcess method) took 164 ms (0 s)
2015-07-06 11:49:36,580 [Dispatcher_req_87867] INFO Dispatcher - Processing 1 request(s)
Updated by Brinette Pierre-Emmanuel over 9 years ago
Staging de 1620 fichiers sur une bande T10K-D insérée dans l'ordre :¶
$ query_volume.ksh KT279300
Volume KT279300 found on Subsystem 3 , Storage class 114
VV Storage
Volume Cond Family Class Space Used Max Volume Size
======== ====== ====== ===== ==================== ====================
KT279300 EOM 0 114 6,466,516,418,247 8,571,050,393,287 Repack output tape
$ lsvol -i3 -v KT279300 | sort -n | process_ls_vol.pl - > KT279300.treqs.test $ export TreqsDBHost=ccsvli10 $ trcp.pl -p KT279300.treqs.test -u test01 2015/07/06 12:14:48 WARN> 1620 prestage requests has been inserted for user test01 2015/07/06 12:14:48 WARN> Program succefully ended
Démarrage des stagers :¶
2015-07-06 12:17:06,415 [Stager_QnKT279300_QId151_stagerNo_1] INFO AbstractProcess - Setting status to value STARTED
2015-07-06 12:17:06,415 [Stager_QnKT279300_QId151_stagerNo_1] INFO Stager - Stager Stager_QnKT279300_QId151_stagerNo_1: starting.
2015-07-06 12:17:06,420 [Stager_QnKT279300_QId151_stagerNo_1] INFO MySQLQueueDAO - Mysql access duration (MySQLQueueDAO.processUpdate method) took 0 ms (0 s)
2015-07-06 12:17:06,420 [Stager_QnKT279300_QId151_stagerNo_1] INFO MySQLQueueDAO - Updated queue 151
2015-07-06 12:17:06,423 [Stager_QnKT279300_QId151_stagerNo_1] INFO MySQLReadingDAO - Mysql access duration (MySQLReadingDAO.processUpdate method) took 2 ms (0 s)
2015-07-06 12:17:06,423 [Stager_QnKT279300_QId151_stagerNo_1] INFO Reading - File /hpss3/dcache/cms/hpssdata/000098214CB7343149D1A701C344ED3C7497 in tape KT279300 at, position { position: 1634, offset: 0 }: Started.
2015-07-06 12:17:06,674 [Observer] INFO MySQLBroker - Mysql access duration (MySQLBroker.executeSelect method) took 1 ms (0 s)
2015-07-06 12:17:07,415 [Activator] INFO Activator - Starting stager 2 of 2
2015-07-06 12:17:07,415 [Activator] INFO AbstractProcess - Setting status to value STARTING
2015-07-06 12:17:07,415 [Activator] INFO Activator - Sleeping between stagers, 1000 millis
2015-07-06 12:17:07,416 [Stager_QnKT279300_QId151_stagerNo_2] INFO AbstractProcess - Setting status to value STARTED
2015-07-06 12:17:07,416 [Stager_QnKT279300_QId151_stagerNo_2] INFO Stager - Stager Stager_QnKT279300_QId151_stagerNo_2: starting.
2015-07-06 12:17:07,423 [Stager_QnKT279300_QId151_stagerNo_2] INFO MySQLQueueDAO - Mysql access duration (MySQLQueueDAO.processUpdate method) took 2 ms (0 s)
2015-07-06 12:17:07,423 [Stager_QnKT279300_QId151_stagerNo_2] INFO MySQLQueueDAO - Updated queue 151
2015-07-06 12:17:07,427 [Stager_QnKT279300_QId151_stagerNo_2] INFO MySQLReadingDAO - Mysql access duration (MySQLReadingDAO.processUpdate method) took 3 ms (0 s)
2015-07-06 12:17:07,427 [Stager_QnKT279300_QId151_stagerNo_2] INFO Reading - File /hpss3/dcache/cms/hpssdata/0000F1BBDAC3130F4D2B87D58B58099BB7BD in tape KT279300 at, position { position: 1199, offset: 0 }: Started.
PROBLEMES¶
Les stagers ont démarrés avec des fichier en position 1199 et 1634 alors qu'il n'y a qu'1 seule queue contenant tous les fichiers de la bande !
Les stages auraient du démarraer en * position 1 et 2*
Updated by Brinette Pierre-Emmanuel over 9 years ago
Nouveaux tests¶
Utilisation du package RPM 'treqs-delivery-1.0-20150707.132153-37.rpm'
Installation¶
pm -Uvh treqs-delivery-1.0-20150707.132153-37.rpm
Preparing... ########################################### [100%]
package treqs-1.7-0.x86_64 is already installed
file /opt/treqs-1.7.0/etc/version.txt from install of treqs-1.7-0.x86_64 conflicts with file from package treqs-1.7-0.x86_64
file /opt/treqs-1.7.0/lib/treqs-java-1.0-SNAPSHOT.jar from install of treqs-1.7-0.x86_64 conflicts with file from package treqs-1.7-0.x86_64
ccsvli10_root# rpm -Uvh treqs-delivery-1.0-20150707.132153-37.rpm --force
Preparing... ########################################### [100%]
1:treqs ########################################### [100%]
Test avec les fichiers de la bande KT279300¶
$ trcp.pl -p KT279300.treqs.test -u test01 2015/07/07 16:10:53 WARN> 1620 prestage requests has been inserted for user test01 2015/07/07 16:10:53 WARN> Program succefully ended
Updated by Brinette Pierre-Emmanuel over 9 years ago
Installation du RPM¶
VERSION: treqs-delivery-1.0-20150713.130035-41.rpm
rpm -Uvh treqs-delivery-1.0-20150713.130035-41.rpm Preparing... ########################################### [100%] 1:treqs ########################################### [100%]
Configuration¶
Mise a jour du schema de la base :¶
# mysql -uroot -pxxxxxxx treqsjobs < CreateDBTables.sql ERROR 1217 (23000) at line 25: Cannot delete or update a parent row: a foreign key constraint fails
Actions