User Tools

Site Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
mission:log:2012:10:25:geoip-database-magic-safety-update-script [2012-10-25 20:51] – created chronomission:log:2012:10:25:geoip-database-magic-safety-update-script [2013-06-30 12:41] (current) – [update-geoip.sh] chrono
Line 1: Line 1:
 +====== GeoIP Database Magic-Safety-Updater ======
 +
 +In the past several months it sometimes just so happened, that the geoipupdate tool, available on many GNU/Linux systems to update MaxMind's GeoIP databases just failed. It exited fine but left a corrupt database, unreadable to the application depending on it. In this particular environment, GeoIP lookup is a mission-critical dependency, so it was time to come up with a little cron/logger/geoipupdate assisted bash magic, to update MaxMind's GeoIP Databases automatically and with two fallback safeties, in case the new DB is corrupt or the download failed somehow.
 +
 +It performed very well for the last several weeks, never raised an alert and is published now, as it might be useful for someone else out there, confronted with the same problem. You could spend your time elsewhere, instead of re-inventing something that is already here and which doesn't rely on alerts, to get a human's attention but tries to fix it by itself in an automated process:
 +
 +===== =====
 +
 +===== update-geoip.sh =====
 +
 +**Dependencies**
 +
 +  - BASH
 +  - logger 
 +  - geoip-bin/libgeoip1
 +
 +<WRAP round tip>
 +With the update-$whatever naming convention, which can be found on most distributions today, naming it //update-geoip.sh// will be an easy to remember TAB-completion.
 +</WRAP>
 +
 +<sxh bash>
 +#!/bin/bash
 +#
 +##########################################################################
 +#
 +#   This program is free software: you can redistribute it and/or modify
 +#   it under the terms of the GNU General Public License as published by
 +#   the Free Software Foundation, either version 3 of the License, or
 +#   (at your option) any later version.
 +#
 +#   This program is distributed in the hope that it will be useful,
 +#   but WITHOUT ANY WARRANTY; without even the implied warranty of
 +#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 +#   GNU General Public License for more details.
 +#
 +#   You should have received a copy of the GNU General Public License
 +#   along with this program. If not, see <http://www.gnu.org/licenses/>.
 +#
 +##########################################################################
 +
 +##########################################################################
 +#   GeoIP Database Magic-Safety Updater
 +##########################################################################
 +#
 +#   The purpose of this script is to keep GeoIP Databases automagically 
 +#   up-to-date and to verify each update. In order to prevent failing 
 +#   frontend apps, which depend on a working GeoIP lookup, the script
 +#   will roll back to a last known good version in case the new DBs 
 +#   fail to geolocate a predefined Test-IP/location combination. 
 +#
 +##########################################################################
 +#   Usage: update-geoip.sh (-v for verbose magic)
 +##########################################################################
 +#
 +#   V1.0   Initial Version                     20120510   chrono
 +#   V1.1   Bugfix - check if .archive exists   20120729   chrono
 +#   V1.2   Public Release under GPL3+          20121024   chrono
 +#
 +
 +G_HOME="/usr/share/GeoIP"     # Home of your GeoIP DB
 +G_BIN="/usr/bin/geoiplookup"  # path to lookup tool
 +G_UPD="/usr/bin/geoipupdate"  # path to update tool
 +G_LOG="/usr/bin/logger"       # path to logger bin
 +G_IP="193.99.144.85"          # Test IP to geolocate
 +G_CHK="DE"                    # Expected geoloc for Test IP 
 +DEBUG=0                       # Set to 1 for debug (or use -v)
 +
 +### functions ############################################################
 +
 +function geo_check()
 +{
 +  local RES=$(${G_BIN} ${G_IP}|sed -n 2p|awk '{print substr($6,0,2)}')
 +  echo ${RES}
 +}
 +
 +function log()
 +{
 +  if [ ${DEBUG} == 1 ]; then
 +    echo "$1"
 +  else
 +    ${G_LOG} "$1"
 +  fi
 +}
 +
 +### prep rollback safety #################################################
 +
 +if [ ! -d "${G_HOME}/.archive/" ]; then
 +  mkdir -p ${G_HOME}/.archive/
 +fi
 +
 +cd ${G_HOME}
 +cp ${G_HOME}/GeoIPCity.dat ${G_HOME}/.archive/GeoIPCity.tmp
 +
 +### try update ###########################################################
 +
 +if [ "$1" == "-v" ]; then
 +  DEBUG=1
 +  ${G_UPD}
 +else
 +  ${G_UPD} > /dev/null 2>&1
 +fi
 +
 +### check update #########################################################
 +
 +CHECK=$(geo_check)
 +
 +if [ "$CHECK" == "$G_CHK" ]; then
 +
 +  ### check successfull - move last known working .dat to archive ########
 +
 +  log "GeoIP DB is valid - Moving last version into archive"
 +  mv ${G_HOME}/.archive/GeoIPCity.tmp ${G_HOME}/.archive/GeoIPCity.dat
 +
 +else
 +
 +  ### check unsuccessfull - rolling back #################################
 +
 +  log "GeoIP DB check FAILED - Rolling back last known working"
 +  mv ${G_HOME}/.archive/GeoIPCity.tmp ${G_HOME}/GeoIPCity.dat
 +
 +  ### check rollback #####################################################
 +
 +  CHECK=$(geo_check)
 +
 +  if [ "$CHECK" == "$G_CHK" ]; then
 +
 +    ### rollback successfull #############################################
 +
 +    log "GeoIP DB Rollback is valid"
 +
 +  else
 +
 +    ### rollback data is corrupt too - move archived version ############# 
 +
 +    log "GeoIP DB Rollback check FAILED - Moving back archive"
 +    mv ${G_HOME}/.archive/GeoIPCity.dat ${G_HOME}/GeoIPCity.dat
 +
 +    ### check archive ####################################################
 +
 +    CHECK=$(geo_check)
 +
 +    if [ "$CHECK" == "$G_CHK" ]; then
 +
 +      ### Archive recovery successfull ###################################
 +
 +      log "Archive recovery OK - phew, that was a close call"
 +
 +    else
 +
 +      ### ALERT - we have no GeoIP capability on this machine ############
 +
 +      log "Holy DevOp... I have failed you and had to kill myself!"
 +
 +      # Do some alerting here if you really depend on working geoip-lookup
 +
 +      exit 1
 +
 +    fi
 +  fi
 +fi
 +
 +exit 0
 +#EOF
 +</sxh>
 +
 +<WRAP round download>
 +**Download the script:**\\
 +[[https://apollo.open-resource.org/downloads/update-geoip.sh.txt|update-geoip.sh.txt ]]
 +</WRAP>
 +
 +{{tag>linux geoip devops software}}
 +
 +{{keywords>Apollo-NG apollo next generation hackerspace hacker space development makerspace fablab diy community open-resource open resource mobile hackbus linux geoip devops software bash geoipupdate geoiplookup geo ip script}}
 +
 +
 +~~DISCUSSION~~
  
Trace:

Page Tools