[Anthill-pro] What CPU usage is expected by Anthill server?
Ryan Smith
rws at urbancode.com
Thu May 22 10:10:40 CDT 2008
I agree, it is unusual. We are testing the idle CPU usage with a large number of agents today. Is the process doing a lot of disk IO? What database are you using, how is its CPU usage?
Ryan
Peter Steele wrote:
> I just applied this change to my 3.5.0 install and restarted the server.
> It's showing under 50MB memory usage right now, which is not surprising
> since I just rebooted the server. It also shows that there are zero
> checked out threads. However, the Anthill Java process still shows up
> as the number 1 CPU consumer, even though no builds nor anything else
> are going on right now. The process is essentially never truly idle;
> it's always churning away doing something. This just doesn't seem right.
> Unfortunately I had not noted what the CPU usage was prior to upgrading
> to 3.5.0 so I don't know if this is something that was introduced by the
> upgrade. Is high CPU usage normal, even when no real activity is going on?
>
>
>
> -----Original Message-----
> From: anthill-pro-bounces at lists.urbancode.com
> [mailto:anthill-pro-bounces at lists.urbancode.com] On Behalf Of Ryan Smith
> Sent: Thursday, May 22, 2008 6:41 AM
> To: Yanko, Curtis
> Cc: AnthillPro user and support list.
> Subject: Re: [Anthill-pro] What CPU usage is expected by Anthill server?
>
>
>
> Everyone should be aware that the memory display was fixed in 3.5.1.
> Prior versions did not correctly display the memory.
>
>
>
> The bottom of
> opt/tomcat/webapps/ROOT/WEB-INF/jsps/currentActivity/memory.jsp should be:
>
>
>
>
>
>
>
> <br>
>
> <div class="note">
>
> <b>Memory:</b><br>
>
> <%
>
> Runtime runtime = Runtime.getRuntime();
>
> long tm = runtime.totalMemory();
>
> long fm = runtime.freeMemory();
>
> long mm = runtime.maxMemory();
>
> long um = tm - fm;
>
> %>
>
> Using <%= getNearestBytes((double) um) %> / <%=
> getNearestBytes((double) tm) %> bytes<br>
>
> <%= getNearestBytes((double) fm) %> bytes free<br>
>
> <%= getNearestBytes((double) mm) %> max memory<br>
>
> </div>
>
>
>
>
>
> Ryan
>
>
>
>
>
> Yanko, Curtis wrote:
>
> > Take a look at the resources tab and see how you are doing on memory.
>
> > When it gets fully consume AHP BMS does some odd things. We have taken
>
> > to upping the JVM memory size. Looking at my own BMS system on Windows
>
> > it is a good, idle app waiting for work. (3.4.11)
>
> >
>
> >
>
> > ===
>
> > -Curt
>
> > W: 860.702.9059
>
> > M: 860.881.2050
>
> >
>
> > -----Original Message-----
>
> > From: anthill-pro-bounces at lists.urbancode.com
>
> > [mailto:anthill-pro-bounces at lists.urbancode.com] On Behalf Of Peter
>
> > Steele
>
> > Sent: Wednesday, May 21, 2008 9:10 PM
>
> > To: rws at urbancode.com; AnthillPro user and support list.
>
> > Subject: RE: [Anthill-pro] What CPU usage is expected by Anthill server?
>
> >
>
> > I agree of course that CPU usage is proportional to activity, but why
>
> > would the Anthill server be consistently the highest CPU consumer, even
>
> > at times when the system is essentially idle--no builds running, no
>
> > cleanup processes running, no codestation resolves (we don't use this
>
> > feature) and no UI activity (beyond anyone with windows open to the
>
> > dashboard). I assume the server is constantly monitoring the agents,
>
> > even if the agents are idle, but that should be the bulk of the
>
> > activity. Are there figures on how well the Anthill server scales? We
>
> > have currently have 50 agents configured, and this number will grow to
>
> > 200+ over the next few months. Can the Anthill server handle this many
>
> > agents?
>
> >
>
> > -----Original Message-----
>
> > From: anthill-pro-bounces at lists.urbancode.com
>
> > [mailto:anthill-pro-bounces at lists.urbancode.com] On Behalf Of Ryan Smith
>
> > Sent: Wednesday, May 21, 2008 5:56 PM
>
> > To: AnthillPro user and support list.
>
> > Subject: Re: [Anthill-pro] What CPU usage is expected by Anthill server?
>
> >
>
> > Peter,
>
> >
>
> > The CPU usage is going to be proportional to the activity on the server.
>
> > Activity includes running workflows, miscellaneous tasks like cleanup,
>
> > UI activity and any remote Codestation resolves. There are other things
>
> > that could cause high CPU usage such as memory thrashing. We are
>
> > investigating potential memory problems with 3.5. If we find one, that
>
> > could be the cause.
>
> >
>
> >
>
> > Ryan Smith
>
> >
>
> > Peter Steele wrote:
>
> >> Here's an interesting snapshot: This is the first few lines from the
>
> >> "top" command:
>
> >>
>
> >>
>
> >>
>
> >> top - 15:26:49 up 64 days, 2:14, 3 users, load average: 0.10, 0.20,
>
> > 0.25
>
> >> Tasks: 143 total, 1 running, 142 sleeping, 0 stopped, 0 zombie
>
> >>
>
> >> Cpu(s): 19.8% us, 0.2% sy, 0.0% ni, 79.9% id, 0.0% wa, 0.0% hi,
>
> > 0.2% si
>
> >> Mem: 4109408k total, 3990708k used, 118700k free, 89004k
>
> > buffers
>
> >> Swap: 8385920k total, 144k used, 8385776k free, 3012000k
>
> > cached
>
> >>
>
> >>
>
> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
>
> >> COMMAND
>
> >>
>
> >> * 7778 root 18 0 530m 300m 8972 S 40 7.5 238:02.05
>
> >> java
>
> > *
>
> >> 30397 root 16 0 7348 1068 772 R 0 0.0 0:29.29
>
> >> top
>
> >>
>
> >> 1 root 16 0 4756 556 460 S 0 0.0 0:01.23 init
>
> >>
>
> >>
>
> >> 2 root RT 0 0 0 0 S 0 0.0 0:01.91
>
> >> migration/0
>
> >>
>
> >>
>
> >>
>
> >> As you can see, process 7778 is current the top CPU user. This is the
>
> >> Anthill server. I rebooted the server just two days ago and it as
>
> >> already consumed over 238 minutes of CPU time. Nothing else comes
>
> > close
>
> >> to this, and the system has been up for 64 days. This seems to be
>
> >> pointing to some kind of problem. What should we expect as far as CPU
>
> >> usage is concerned for the Anthill server process?
>
> >>
>
> >>
>
> >>
>
> >> *From:* anthill-pro-bounces at lists.urbancode.com
>
> >> [mailto:anthill-pro-bounces at lists.urbancode.com] *On Behalf Of *Peter
>
> > Steele
>
> >> *Sent:* Wednesday, May 21, 2008 1:21 PM
>
> >> *To:* AnthillPro user and support list.
>
> >> *Subject:* RE: [Anthill-pro] What CPU usage is expected by Anthill
>
> > server?
>
> >>
>
> >>
>
> >> The server still seems to be a CPU hog, with it almost always being
>
> > the
>
> >> number one process on the "top" list, even when the system is largely
>
> >> idol. We've added 40+ agents recently, but I did not check the
>
> > server's
>
> >> CPU usage before adding all the new agents, and I also did not what
>
> > the
>
> >> CPU usage was before upgrading to 3.5. Fortunately the command
>
> > failures
>
> >> we were seeing the other day appear to have stopped after the reboot,
>
> >> although the GUI is still very slow to respond to many typical
>
> > commands.
>
> >> For example, it just took 30 seconds for Anthill to respond to a click
>
> >
>
> >> on a project displayed in the "Your Active Projects" column of the
>
> >> Dashboard. This seems to be slower than what it was in 3.4.12, but as
>
> > I
>
> >> indicated we have added a lot of agents lately. We'll be adding more
>
> > as
>
> >> well, exceeding 100 agents in the next while. Is there anything we
>
> >> should do to improve performance?
>
> >>
>
> >>
>
> >>
>
> >> *From:* anthill-pro-bounces at lists.urbancode.com
>
> >> [mailto:anthill-pro-bounces at lists.urbancode.com] *On Behalf Of *Steve
>
> > Boone
>
> >> *Sent:* Wednesday, May 21, 2008 9:39 AM
>
> >> *To:* AnthillPro user and support list.
>
> >> *Subject:* Re: [Anthill-pro] What CPU usage is expected by Anthill
>
> > server?
>
> >>
>
> >>
>
> >> Peter,
>
> >>
>
> >> Has this been happening in a regular basis?
>
> >>
>
> >> What happened after the restart?
>
> >>
>
> >> On Wed, May 14, 2008 at 6:44 PM, Peter Steele <psteele at maxiscale.com
>
> >> <mailto:psteele at maxiscale.com>> wrote:
>
> >>
>
> >> We've noticed some very slow feedback from the Anthill server today,
>
> >> even coming back with failures for some operations. The Unix top
>
> > command
>
> >> is showing the ah3server process as taking as much as 100% CPU at
>
> > times,
>
> >> and is consistently the top 1 or 2 processes on the system. There are
>
> >> just two builds going on, and both of course are running on agent
>
> >> systems, not the server. I don't recall seeing this before. We
>
> > upgraded
>
> >> to 3.5 yesterday and I am wondering if that might be to blame. It's
>
> >> certainly very slow right now; I'll restart the server as soon as the
>
> >> builds are done...
>
> >>
>
> >>
>
> >>
>
> >>
>
> >> _______________________________________________
>
> >> Anthill-pro mailing list
>
> >> Anthill-pro at lists.urbancode.com
>
> > <mailto:Anthill-pro at lists.urbancode.com>
>
> >> http://lists.urbancode.com/mailman/listinfo/anthill-pro
>
> >>
>
> >>
>
> >>
>
> >>
>
> >>
>
> > ------------------------------------------------------------------------
>
> >> _______________________________________________
>
> >> Anthill-pro mailing list
>
> >> Anthill-pro at lists.urbancode.com
>
> >> http://lists.urbancode.com/mailman/listinfo/anthill-pro
>
> >
>
> > --
>
> > ===========================================================
>
> > Ryan Smith. 2044 Euclid Ave., Suite 600
>
> > Developer Cleveland, Ohio 44115
>
> > Urbancode, Inc.
>
> > email: rws at urbancode.com
>
> > web: www.urbancode.com phone: 216-858-9000
>
> > web: www.anthillpro.com fax: 216-858-9602
>
> > ===========================================================
>
> > _______________________________________________
>
> > Anthill-pro mailing list
>
> > Anthill-pro at lists.urbancode.com
>
> > http://lists.urbancode.com/mailman/listinfo/anthill-pro
>
> > _______________________________________________
>
> > Anthill-pro mailing list
>
> > Anthill-pro at lists.urbancode.com
>
> > http://lists.urbancode.com/mailman/listinfo/anthill-pro
>
> >
>
> >
>
> > This e-mail, including attachments, may include confidential and/or
>
> > proprietary information, and may be used only by the person or entity to
>
> > which it is addressed. If the reader of this e-mail is not the intended
>
> > recipient or his or her authorized agent, the reader is hereby notified
>
> > that any dissemination, distribution or copying of this e-mail is
>
> > prohibited. If you have received this e-mail in error, please notify the
>
> > sender by replying to this message and delete this e-mail immediately.
>
> >
>
> >
>
>
>
> --
>
> ===========================================================
>
> Ryan Smith. 2044 Euclid Ave., Suite 600
>
> Developer Cleveland, Ohio 44115
>
> Urbancode, Inc.
>
> email: rws at urbancode.com
>
> web: www.urbancode.com phone: 216-858-9000
>
> web: www.anthillpro.com fax: 216-858-9602
>
> ===========================================================
>
> _______________________________________________
>
> Anthill-pro mailing list
>
> Anthill-pro at lists.urbancode.com
>
> http://lists.urbancode.com/mailman/listinfo/anthill-pro
>
--
===========================================================
Ryan Smith. 2044 Euclid Ave., Suite 600
Developer Cleveland, Ohio 44115
Urbancode, Inc.
email: rws at urbancode.com
web: www.urbancode.com phone: 216-858-9000
web: www.anthillpro.com fax: 216-858-9602
===========================================================
More information about the Anthill-pro
mailing list