CLOUDSTACK-6989: Add 3 strikes rule for RvR freezing detection

Sometime in VR ntpd would move time backward to keep sync with NTP server, which
can result in false alarm of keepalived monitering process.

This patch adds 3 strikes for keepalived process dead detection to avoid falsely
shutdown keepalived process due to time adjustment for only once.
This commit is contained in:
Sheng Yang 2014-06-24 16:58:55 -07:00
parent f756d4aa33
commit 75c9a20c77

View File

@ -17,12 +17,27 @@
# under the License.
STRIKE_FILE="[RROUTER_BIN_PATH]/keepalived.strikes"
if [ -e [RROUTER_BIN_PATH]/keepalived.ts2 ]
then
lasttime=$(cat [RROUTER_BIN_PATH]/keepalived.ts2)
thistime=$(cat [RROUTER_BIN_PATH]/keepalived.ts)
diff=$(($thistime - $lasttime))
s=0
if [ $diff -lt 30 ]
then
if [ -e $STRIKE_FILE ]
then
s=`cat $STRIKE_FILE 2>/dev/null`
fi
s=$(($s+1))
echo $s > $STRIKE_FILE
else
rm $STRIKE_FILE
fi
#3 strikes rule
if [ $s -gt 2 ]
then
echo Keepalived process is dead! >> [RROUTER_LOG]
[RROUTER_BIN_PATH]/services.sh stop >> [RROUTER_LOG] 2>&1