RTFACT-5355

Document Sample
RTFACT-5355 Powered By Docstoc
					[RTFACT-5355] Error During Garbage Collection Created: 02/Jan/13
 Updated: 09/Jan/13
Status:          Open
Project:         Artifactory Binary Repository
Component/s:     Artifact Storage
Affects
                 None
Version/s:
Fix Version/s:   None

Type:            Bug                        Priority:         Normal
Reporter:        Shawn Ayers                Assignee:         Unassigned
Resolution:      Unresolved                 Votes:            1
Labels:          None
Remaining        Not Specified
Estimate:
Time Spent:      Not Specified
Original         Not Specified
Estimate:

Attachments:         output.txt

 Descriptio
    n
We are seeing the following errors from one of our Artifactory servers:

2013-01-02 14:41:32,212 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/263555.'
2013-01-02 14:41:33,339 [http-8080-2] [WARN ] (o.a.r.ArtifactoryResponseBase:52) -
Zero bytes sent to client.
2013-01-02 14:41:57,238 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/263555'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-02 14:41:59,846 [art-exec-237] [INFO ] (o.a.a.r.c.r.RemoteReplicator:67) -
Completed remote folder replication for 'v150_Packages:' with 0 delete, 0 download and
0 mkdir events.
2013-01-02 14:42:00,463 [http-8080-2] [WARN ] (o.a.r.ArtifactoryResponseBase:52) -
Zero bytes sent to client.
2013-01-02 14:42:12,633 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/446707.'




 Comment
    s
Comment by Shay Yaakov [ 03/Jan/13 ]
Hi Shawn,

Just to verify, we are not talking on one of the Artifactory Onlines right?
Are you running on the latest Artifactory version? (2.6.6 but if not at least > 2.6.3)?
One additional question – do the same errors still appear regularly after this first time?
Comment by Shawn Ayers [ 04/Jan/13 ]
Shay,

Sorry for the delay. Correct, this is not one of the cloud servers. No, we have not
upgraded to the latest version, but are running 2.6.5. We are still seeing the errors and the
machine is consuming most of the CPU processes.

Not quite sure what is going on with this machine at the moment.
Comment by Shawn Ayers [ 04/Jan/13 ]
We are thinking that these warnings may be causing the spike in CPU utilization. The log
is filled with these:

9 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting force removal of parent
node at '/trash/155281.'
2013-01-04 11:23:42,994 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/155281'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:43,705 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/26131.'
2013-01-04 11:23:45,271 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/26131'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:46,009 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/416645.'
2013-01-04 11:23:47,424 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/416645'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:47,987 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/160025.'
2013-01-04 11:23:49,594 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/160025'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:50,443 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/468902.'
2013-01-04 11:23:50,998 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/468902'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:51,649 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/73225.'
2013-01-04 11:23:52,286 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/73225'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:53,149 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/645360.'
2013-01-04 11:23:53,814 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/645360'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:54,371 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/360212.'
2013-01-04 11:23:54,927 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/360212'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:55,808 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/974850.'
2013-01-04 11:23:57,543 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/974850'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:23:58,497 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/642430.'
2013-01-04 11:24:00,253 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/642430'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:24:01,257 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/43419.'
2013-01-04 11:24:03,354 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/43419'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:24:04,519 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/192834.'
2013-01-04 11:24:06,260 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/192834'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:24:07,381 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/281769.'
2013-01-04 11:24:11,804 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/281769'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:24:13,299 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/960139.'
2013-01-04 11:24:17,711 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/960139'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:24:19,358 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/263596.'
2013-01-04 11:24:23,259 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :481) - Cannot
complete force removal of node at '/trash/263596'.: aa936b90-66d0-4f37-813a-
bc25cb24989c.
2013-01-04 11:24:25,360 [art-exec-1] [WARN ] (o.a.j.u.JcrHelper :473) - Attempting
force removal of parent node at '/trash/210457.'
Comment by Shay Yaakov [ 04/Jan/13 ]
I've sent you a detailed email.
Comment by Shawn Ayers [ 09/Jan/13 ]
Shay,

We have successfully ran the curl scripts, but there still appears to be nodes available for
deletion.

We have run the fix with the following output:

[skayers@can64ayerstest2 ~]$ curl -X POST -i -u USER:PASSWORD
"http://watatsrv01:8080/artifactory/api/plugins/execute/cleanTrash?params=fix=true"
HTTP/1.1 200 OK
Server: Artifactory/2.6.6
X-Artifactory-Id: 516318cb0988e592:2d4ee3c0:13c1c4d5a23:-8000
Content-Type: text/plain
Transfer-Encoding: chunked
Date: Wed, 09 Jan 2013 15:46:00 GMT

Fix consistency of [13821d7e-5b4f-4865-95db-9795b9e38d05] for [default]
[skayers@can64ayerstest2 ~]$

Then if I run the following script to check for nodes to delete:

curl -X POST -i -u USER:PASSWORD
http://watatsrv01:8080/artifactory/api/plugins/execute/cleanTrash >
/home/staff/skayers/output.txt

There are still several nodes that can be deleted.

I have also attached the output.txt file for your review.

My other question, is why are we seeing this trash/node issue only on this particular
server?

Thank you,

Shawn.
Generated at Sun Jun 22 18:08:39 UTC 2014 using JIRA 6.0.1#6096-
sha1:e4a48bd73c6b8a4d99c824976ce5808b4c85857d.

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:7/17/2014
language:English
pages:4