Running Monyog On Aws Ec2

[cloudcontrol]

Hello,

We have an installation and are very happy with the features, but we are having difficulty with our particular use case. The installation is the 64-bit version of MonYOG Ultimate 4.6.2 on Ubuntu Lucid 10.04.3 64-bit AWS EC2 c1.xlarge instance (7.5GB RAM and 8 cores with the approximate power of a 2.4GHz 2007 Opteron CPU). We have an unlimited license.

Our use case is that we have monYOG monitoring 12 AWS RDS instances, 4 with collection interval set to 1 minute, no sniffer enabled, and 8 production RDS instances with collection interval set to 10 seconds, no sniffer enabled.

We typically have MonYOG running in the Chrome browser on Windows 7 with six of the production servers selected and 3 tabs open (1) the dashboard tab showing the last 15 intervals, (2) the monitors/advisors tab showing deltas for Current Connections and (3) the monitors/advisors tab showing deltas for Statements.

Attached is a screenshot of the cloudwatch stats which show a flat line 50% CPU usage over the last three hours and 600 i/o ops on the “hard disk”- that's about the max i/o ops you can sustain on and AWS EC2 instance. I could do some disk striping but first wanted to verify which application parameters might decrease the disk i/o ops.

Thanks!

[Mahesh]

which application parameters might decrease the disk i/o ops ?

— I think you here you can increase Data collection interval to 5 minutes

which is default?

Also why you want to have Data collection = 10 second on a Production server ?

If you increase to 5 minutes definitely you will see significant decrease on Disk I/o operations.

Also another point is consistently viewing Monitors/Advisers page, Refreshing it frequently?

You can of course want to see stats about your server at any time,viewing less frequently or refreshing page frequently might decrease Disk I/O operations.

MONyog resolves each counter when you are viewing on Monitors/Adviser page.

We are working on optimizing resolving MONyog counters.

[cloudcontrol]

'Mahesh' wrote:

which application parameters might decrease the disk i/o ops ?

— I think you here you can increase Data collection interval to 5 minutes

which is default?

Also why you want to have Data collection = 10 second on a Production server ?

If you increase to 5 minutes definitely you will see significant decrease on Disk I/o operations.

Thank you- 5 minutes is a bit too slow to react to problems on our social networking site. We are now using 1 minute as the interval and have seen performance improve, though still slow to “load up” the monitor/advisor page initially. Regarding load MonYOG puts on our database: I am not sure I understand your concern about a 10 second interval on a production server- is monyog performing selects or simply monitoring the process list? Given the hardware specs I posted, would you imagine a 10 second interval is seriously taxing the system?

'Mahesh' wrote:

Also another point is consistently viewing Monitors/Advisers page, Refreshing it frequently?

You can of course want to see stats about your server at any time,viewing less frequently or refreshing page frequently might decrease Disk I/O operations.

MONyog resolves each counter when you are viewing on Monitors/Adviser page.

We are working on optimizing resolving MONyog counters.

I see. No, we are not refreshing, because each page refresh takes a great deal of time to load all of the counters in monitors/advisors and we prefer to watch the counters update automatically.

Thanks for your help!

[Mahesh]

Code:

Thank you- 5 minutes is a bit too slow to react to problems on our social networking site. We are now using 1 minute as the interval and have seen performance improve, though still slow to “load up” the monitor/advisor page initially. Regarding load MonYOG puts on our database: I am not sure I understand your concern about a 10 second interval on a production server- is monyog performing selects or simply monitoring the process list? Given the hardware specs I posted, would you imagine a 10 second interval is seriously taxing the system?

— It's upto your requirement what data collection time you want to set depending on application.

Considering your environment and fact that sniffer is disabled too, I think 10 second interval should not be a problem

as MONyog just gets stats from server using Show global status and show global variables.

Monitors/Advisors page initially takes time to show counters value because MONyog resolves those counters dynamically and shows in page later you can see there is no delay in showing stats for any counter, only during first time you will see a message like “Getting details..” for few seconds.

Also getting processlist information is involved only if you enable Sniffer –> Processlist option, otherwise MONyog won't collect processlist information.

What I meant is it's of course that 1 second load is GREATER THAN 10 seconds GREATER THAN 1 minute etc..

Still considering your environment 1 seconds interval also MONyog can run without any problem.

Basically MONyog is a very light weight app also we have done some benchmarking and here is detail about it:

Code:

OS Spec: Linux 2.6.9-67.ELsmp #1 SMP x86_64 x86_64 GNU/Linux

System Spec:
CPU model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
stepping : 10
cpu MHz : 2992.388
Total Memory : 256MB

After Analysis of Zero, One , Two, Five and Ten Server.

Average rate has been calculated as:

CPU remains constant to 0

Only MYSQL is Enabled:

Average Memory used in %/Server = 0.5/server

Average Virtual Memory used/Server = 10m/Server

Resident memory used/Server = n/a

Only MySQL and Sniffer is enabled:

Average Memory used in %/Server = 1.0/server

Average Virtual Memory used/Server = 25m/Server

Resident memory used/Server = 1.5m/server

Only MySQL, Sniffer and OS is enabled:

Average Memory used in %/Server = 1.0/server

Average Virtual Memory used/Server = 32m/Server

Resident memory used/Server = 2.2m/server

Only MySQL, Sniffer, OS and notification is enabled:

Average Memory used in %/Server = 1.0/server

Average Virtual Memory used/Server = 32m/Server

Resident memory used/Server = 2.2m/server

Now Average report when Only Sniffer, OS and Notification is enabled at a time:

Only MYSQL is Enabled:

Average Memory used in %/Server = 0.5/server

Average Virtual Memory used/Server = 10m/Server

Only MYSQL and Sniffer is Enabled:

Average Memory used in %/Server = 1.0/server

Average Virtual Memory used/Server = 30m/Server

Resident memory used/Server = 1.5m/server

Only MYSQL and OS is Enabled:

Average Memory used in %/Server = 0.6/server

Average Virtual Memory used/Server = 21m/Server

Resident memory used/Server = 1m/server

Only MYSQL and Notification is Enabled:

Average Memory used in %/Server = 0.6/server

Average Virtual Memory used/Server = 11m/Server

Resident memory used/Server = 1m/server

(I think you might get some idea from this report)

Code:

I see. No, we are not refreshing, because each page refresh takes a great deal of time to load all of the counters in monitors/advisors and we prefer to watch the counters update automatically.

— As I have mentioned earlier initially on page load MONyog shows “Getting details for server…” after that do not have to refresh screen multiple times you will see updates automatically after specified data collection, you can verify by click on chart as you see chart gets updated.

If any further question please get back to us

Thanks,

Mahesh

[Mahesh]

You may refer this blog post:

http://www.webyog.com/blog/2011/07/20/what-are-hardware-requirements-for-monyog/

[peterlaursen]

Further to get closer to an understanding of this, we will need to understand the complete 'deployment secenario' – ie. what else is running on the same machine as MONyog? It could be some other application saturation the I/O capabilities of the system.

Also ahve you tried following MONyog's advisors for setting various cache/buffer sizes (in server level and engine level)?

Also some of the issues may be networking problems between the MONyog machine and the machine where you view it.

[cloudcontrol]

'Mahesh' wrote:

You may refer this blog post:

http://www.webyog.com/blog/2011/07/20/what-are-hardware-requirements-for-monyog/

Thanks- that is one of the better requirements pages I have seen. Seems like we should be OK on resources, but I went ahead and created a RAID 0 array of 10 EBS volumes just to make sure. I have the monyog root symlinked to a directory on the striped array and still have not seen a performance increase.

Unfortunately while the tool is adequate for near real time information, it is completely unreliable when I try to look at historical data. Sometimes it works, and other times I have experiences like today: I have been trying for over two hours to view data from a simple Monitors/Advisors History/Trends page for the range 6:30 AM to 7:15 AM on a single server. Only stopping monyog and restarting monyog will allow me to bring the page up completely. Once up, I have twice waiting another 45 minutes for a details page which would give me a minute by minute breakdown of insert statements during that historical period. The RAID has not caused this issue- this problem existed before moving the monyog root folder as well.

Attached is a screenshot of the monyog systems' “htop” command and a cloudwatch screenshot. The cloudwatch screenshot is most interesting- it shows we have excess CPU that is not being utilized, and that disk iops are plateauing at 200/sec. Cloudwatch cannot show us disk iops on the RAID array, unfortunately. There is also 5 GB of RAM available on the system, which is also used for very lightly accessed mediawiki and joomla installation.

Question I have are:

1- is monyog using temp space for reads writes outside the monyog root (which is now running on the raid 0 array?

2- why won't monyog “grab” more system resources?

3- given the specs you linked to, or usage seems quite light- why can't I reliably load that simple historical page?

Thanks for your help!

[Mahesh]

Hi,

Thanks for your information.

I think it will be better to have a screen sharing session to check reported issues.

Can you please create a ticket by sending an email to “[email protected]”

[Author]

Posts