Os Checks Works For 5 Mins Then Stops

[JohanLR]

Hi All,

I just installed MONyog for evaluation to purchase and i have an interesting issue.

Im monitoring 2 servers and for some reason the OS checks (CPU. MEM ,IO etc ) works when the MONyog service starts but around 5 to 10 mins later the Linux OS monitors go red.

 

I created a custom user on the monitored servers with the recommended group memberships , it works but after 5 to 10 mins the monitor goes red.

For shits and giggles i updated the server setups in MONyog to use root to collect the OS data. Restarted the service and the monitors ran again for 5 to 10 mins and went red again 🙁

 

I checked my settings in the SSHD_CONFIG of my monitored servers and root login in allowed and there is no session timeout set.

 

Here is my log:

[6.1] [2014-09-25 14:16:45] [Server: someservername] src/ysshchannel.cpp(59) ErrCode:-2 ErrMsg:SshChannelOpen: Channel start error — Connection closed by remote host

[6.1] [2014-09-25 14:16:45] [Server: someservername] src/ysshchannel.cpp(76) ErrCode:-2 ErrMsg:ChannelRequestExec: SSH channel not initialized

[6.1] [2014-09-25 14:16:45] [Server: someservername] populatesystem.cpp(297) ErrCode:-1 ErrMsg:GetUnixTimeStamp: ChannelRequestExec: SSH channel not initialized

[6.1] [2014-09-25 14:16:45] [Server: someservername] src/ysftp.cpp(163) ErrCode:-2 ErrMsg:SftpFileOpen: Could not open file ‘/proc/vmstat’ in ‘0’ access mode: Connection closed by remote host

[6.1] [2014-09-25 14:16:45] [Server: someservername] populatesystem.cpp(535) ErrCode:-1 ErrMsg:GetFileFromProc: SftpFileOpen: Could not open file ‘/proc/vmstat’ in ‘0’ access mode: Connection closed by remote host

[6.1] [2014-09-25 14:17:26] [Server: someOTHERservername] src/ysshchannel.cpp(59) ErrCode:-2 ErrMsg:SshChannelOpen: Channel start error — Connection closed by remote host

[6.1] [2014-09-25 14:17:26] [Server: someOTHERservername] src/ysshchannel.cpp(76) ErrCode:-2 ErrMsg:ChannelRequestExec: SSH channel not initialized

[6.1] [2014-09-25 14:17:26] [Server: someOTHERservername] populatesystem.cpp(297) ErrCode:-1 ErrMsg:GetUnixTimeStamp: ChannelRequestExec: SSH channel not initialized

[6.1] [2014-09-25 14:17:26] [Server: someOTHERservername] src/ysftp.cpp(163) ErrCode:-2 ErrMsg:SftpFileOpen: Could not open file ‘/proc/vmstat’ in ‘0’ access mode: Connection closed by remote host

[6.1] [2014-09-25 14:17:26] [Server: someOTHERservername] populatesystem.cpp(535) ErrCode:-1 ErrMsg:GetFileFromProc: SftpFileOpen: Could not open file ‘/proc/vmstat’ in ‘0’ access mode: Connection closed by remote host

[6.1] [2014-09-25 14:21:46] [Server: someservername] src/ysshchannel.cpp(59) ErrCode:-2 ErrMsg:SshChannelOpen: Channel start error — Connection closed by remote host

[6.1] [2014-09-25 14:21:46] [Server: someservername] src/ysshchannel.cpp(76) ErrCode:-2 ErrMsg:ChannelRequestExec: SSH channel not initialized

[6.1] [2014-09-25 14:21:46] [Server: someservername] populatesystem.cpp(297) ErrCode:-1 ErrMsg:GetUnixTimeStamp: ChannelRequestExec: SSH channel not initialized

[6.1] [2014-09-25 14:21:46] [Server: someservername] src/ysftp.cpp(163) ErrCode:-2 ErrMsg:SftpFileOpen: Could not open file ‘/proc/vmstat’ in ‘0’ access mode: Connection closed by remote host

[6.1] [2014-09-25 14:21:46] [Server: someservername] populatesystem.cpp(535) ErrCode:-1 ErrMsg:GetFileFromProc: SftpFileOpen: Could not open file ‘/proc/vmstat’ in ‘0’ access mode: Connection closed by remote host

[6.1] [2014-09-25 14:22:27] [Server: someOTHERservername] src/ysshchannel.cpp(59) ErrCode:-2 ErrMsg:SshChannelOpen: Channel start error — Connection closed by remote host

[6.1] [2014-09-25 14:22:27] [Server: someOTHERservername] src/ysshchannel.cpp(76) ErrCode:-2 ErrMsg:ChannelRequestExec: SSH channel not initialized

[6.1] [2014-09-25 14:22:27] [Server: someOTHERservername] populatesystem.cpp(297) ErrCode:-1 ErrMsg:GetUnixTimeStamp: ChannelRequestExec: SSH channel not initialized

[6.1] [2014-09-25 14:22:27] [Server: someOTHERservername] src/ysftp.cpp(163) ErrCode:-2 ErrMsg:SftpFileOpen: Could not open file ‘/proc/vmstat’ in ‘0’ access mode: Connection closed by remote host

[6.1] [2014-09-25 14:22:27] [Server: someOTHERservername] populatesystem.cpp(535) ErrCode:-1 ErrMsg:GetFileFromProc: SftpFileOpen: Could not open file ‘/proc/vmstat’ in ‘0’ access mode: Connection closed by remote host

 

Im also double checked that my firewalls are disabled on all servers in this equation. 

 

The MONyog server and the 2 monitored servers are all Centos7.

 

Your help would be greatly appreciated.

 

 

 

[sandhya.r]

Hi,

 

Can you please tell the value of “ClientAliveInterval” in “/etc/ssh/sshd_config”?

 

Please try increasing the value of “ClientAliveInterval” parameter? the value is in seconds (and the default value is 0). 

The “ClientAliveInterval” value should be greater than the data collection interval of that server(Edit a server –> Advanced Settings –> Data collection interval).

If your data collection is set to 5 minutes (i.e. 300 seconds) the “ClientAliveInterval” can be set to, say, 900 seconds. which is greater than data collection interval specified in MONyog.

 

Also, you will need to set “KeepAlive” to yes. For the changes to be effective you will need to restart your sshd service.

Let us know whether this resolves your problem. 

 

Thanks,

Sandhya

Team MONyog

[JohanLR]

Hi Sandhya,

Thanks for the reply , my ClientAliveInterval was set to 15 but even after disabling it and restarting the SSHD the problem still persists. I changed the data collection interval to 3 mins and its still a problem . Changing the data collection interval to 30 secs seems to work fine but that is not practical.

Also , my environment is PCI-DSS certified so i can’t use KeepAlive. Oddly enough , 1 of the 3 mysql servers that im monitoring does not have this problem even though its SSHD_CONFIG is the same as the others. 

 

Anything else i can take a look at ?

 

Regards,

Johan 

[sandhya.r]

Hi Johan,

 

ClientAliveInterval value is in seconds. If your data collection interval is 3 minutes, we recommend you to set “ClientAliveInterval” as 300 seconds(which is > 3minutes).

 

Thanks,

Sandhya.

[JohanLR]

Thanks for the tip but i already tried that , its still happening even if i disable the ClientAliveInterval setting .

[peterlaursen]

I think there may besome gateway/router setting on your server or network independent of SSHD settings interrupting inactive connections.

[Author]

Posts