Project

General

Profile

Actions

Bug #9695

open

Facing ConcurrentModificationException ?

Added by Chambon Bernard about 7 years ago. Updated about 7 years ago.

Status:
In progress
Priority:
Normal
Assigned To:
Category:
-
Start date:
04/14/2015
Due date:
% Done:

0%

Estimated time:

Description

Let's start with ConcurrentModificationException

  • ConcurrentModificationException
    Found on both servers !, found 3 use cases of such error

use case 1

java.util.ConcurrentModificationException: null
    at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922) ~[na:1.7.0_65]
    at java.util.HashMap$KeyIterator.next(HashMap.java:956) ~[na:1.7.0_65]
    at fr.in2p3.cc.storage.treqs.control.controller.QueuesController.getWaitingQueues(QueuesController.java:626) ~[jtreqs-server-jav
a-1.6.1-SNAPSHOT.jar:na]
    at fr.in2p3.cc.storage.treqs.control.activator.Activator.process(Activator.java:466) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
    at fr.in2p3.cc.storage.treqs.control.activator.Activator.action(Activator.java:251) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
    at fr.in2p3.cc.storage.treqs.control.activator.Activator.toStart(Activator.java:670) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
    at fr.in2p3.cc.storage.treqs.control.process.AbstractProcess.run(AbstractProcess.java:204) [jtreqs-server-java-1.6.1-SNAPSHOT.ja
r:na]
2014-09-21 22:00:25,577 [Activator] WARN  Activator Stopped
JNIException created
2014-09-21 22:00:25,578 [tape-1411329623542-54-JTC61400-27061] WARN  Error processing this file: /hpss/in2p3.fr/group/dchooz/DATA/raw/DC
Run0017577_RecoP_v17_FD_prof_5.root File locked or HSM is currently unavailable.
2014-09-21 22:00:25,578 [tape-1411329623542-54-JTC61400-27061] WARN  General error. Retrying /hpss/in2p3.fr/group/dchooz/DATA/raw/DCRun0
017577_RecoP_v17_FD_prof_5.root
2014-09-21 22:01:21,871 [Dispatcher] WARN  Dispatcher Stopped
2014-09-21 22:02:04,347 [main] ERROR No heartbeat, the application is dying.
2014-09-21 22:02:04,347 [main] WARN  Dispatcher and Activator are stopped
jTReqS-Server started

use case 2
2015-03-11 07:17:23,872 [Dispatcher] ERROR f.i.c.s.t.c.dispatcher.Dispatcher - Stopping
fr.in2p3.cc.storage.treqs.control.dispatcher.DispatcherException: java.util.ConcurrentModificationException
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.action(Dispatcher.java:237) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.toStart(Dispatcher.java:887) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.process.AbstractProcess.run(AbstractProcess.java:204) [jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
Caused by: java.util.ConcurrentModificationException: null
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922) ~[na:1.7.0_75]
        at java.util.HashMap$ValueIterator.next(HashMap.java:950) ~[na:1.7.0_75]
        at fr.in2p3.cc.storage.treqs.control.controller.FilePositionOnTapesController.exists(FilePositionOnTapesController.java:276) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.controller.TapesController.cleanup(TapesController.java:166) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.cleaningReferences(Dispatcher.java:342) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.action(Dispatcher.java:232) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]

use case 3
2015-04-04 06:31:12,608 [Dispatcher] ERROR f.i.c.s.t.c.dispatcher.Dispatcher - Stopping
fr.in2p3.cc.storage.treqs.control.dispatcher.DispatcherException: java.util.ConcurrentModificationException
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.action(Dispatcher.java:255) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.toStart(Dispatcher.java:887) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.process.AbstractProcess.run(AbstractProcess.java:204) [jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
Caused by: java.util.ConcurrentModificationException: null
        at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859) ~[na:1.7.0_75]
        at java.util.ArrayList$Itr.next(ArrayList.java:831) ~[na:1.7.0_75]
        at fr.in2p3.cc.storage.treqs.control.controller.StagersController.getActiveStagersForQueue(StagersController.java:219) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.controller.QueuesController.addFilePositionOnTape(QueuesController.java:200) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.submitRequest(Dispatcher.java:866) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.innerProcess(Dispatcher.java:610) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.process(Dispatcher.java:701) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.retrieveNewRequests(Dispatcher.java:772) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        at fr.in2p3.cc.storage.treqs.control.dispatcher.Dispatcher.action(Dispatcher.java:242) ~[jtreqs-server-java-1.6.1-SNAPSHOT.jar:na]
        ... 2 common frames omitted
2015-04-04 06:31:12,609 [Dispatcher] WARN  f.i.c.s.t.c.dispatcher.Dispatcher - Dispatcher Stopped

Actions #1

Updated by Chambon Bernard about 7 years ago

  • Description updated (diff)
Actions #2

Updated by Chambon Bernard about 7 years ago

  • Description updated (diff)
Actions #3

Updated by Chambon Bernard about 7 years ago

  • Description updated (diff)
Actions #4

Updated by Chambon Bernard about 7 years ago

  • Status changed from New to In progress
  • Using ConcurrentHashMap and/or synchronized(Object) facilities
  • Providing some methods for tests usage (see Dispatcher | Activator classes)
  • Test via TreqsStability code => tests ok
  • Deploy treqs rpm, insert requests => tests ok
  • Memo: Working on git branch "stability-fix"
Actions #5

Updated by Chambon Bernard about 7 years ago

  • Subject changed from Facing Exceptions .... to Facing ConcurrentModificationException ?
Actions

Also available in: Atom PDF