Database and LDAP backup scripts

It is a common task to make backups of database or LDAP on the server. Here are examples of such scripts for Linux.

For MySQL database backup I use this script

#!/bin/sh
export PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin
umask 077

savedays="180"
backupdir="/mnt/backups/db"

now=`date "+%Y%m%d_%H%M"`

if [ ! -d ${backupdir} ] ; then
    echo Creating ${backupdir}
    mkdir -p  ${backupdir}; chmod 700 ${backupdir}; 
fi

mysqldump -hlocalhost -uuser -ppassword --routines database | gzip > ${backupdir}/database_${now}.sql.gz  2>>/var/log/dbbackups.log
find ${backupdir} -name 'database_*' -a -mtime +${savedays} -print -delete

For LDAP backup I use this script:

#!/bin/sh
export PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin
umask 077

savedays="180"
backupdir="/mnt/backups/ldap"

now=`date "+%Y%m%d_%H%M"`

if [ ! -d ${backupdir} ] ; then
    echo Creating ${backupdir}
    mkdir -p  ${backupdir}; chmod 700 ${backupdir}; 
fi
        
slapcat > ${backupdir}/ldap.${now}.ldif  2>>/var/log/ldapbackups.log         
find ${backupdir} -name 'ldap_*' -a -mtime +${savedays} -print -delete

These script will save your database and LDAP backups to /mnt/backups/db and /mnt/backups/ldap accordingly. Backup files will be kept for 180 days and then they will be deleted.

To automate backup you can simply place the scripts to the appropriate cron directory (e.g. /etc/cron.daily) and make sure that cron has rights to execute this file and has rights to write to /mnt/backups/db and /mnt/backups/ldap folders.

Read more...

Getting Search Volume with Google Adwords API (TargetingIdeaService) - new Java library

Google provides an API to get AdWords data, but there is a little amount of examples of its usage. I'll show you simple example how to get demand (search volume) data for specific words using TargetingIdeaService. In previous post I showed how to do it using the old library, in this post we'll use the new library.

Here is the example:

package loader;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

import com.google.api.ads.adwords.axis.factory.AdWordsServices;
import com.google.api.ads.adwords.axis.v201302.cm.Language;
import com.google.api.ads.adwords.axis.v201302.cm.Location;
import com.google.api.ads.adwords.axis.v201302.cm.Money;
import com.google.api.ads.adwords.axis.v201302.cm.Paging;
import com.google.api.ads.adwords.axis.v201302.o.Attribute;
import com.google.api.ads.adwords.axis.v201302.o.AttributeType;
import com.google.api.ads.adwords.axis.v201302.o.IdeaType;
import com.google.api.ads.adwords.axis.v201302.o.LanguageSearchParameter;
import com.google.api.ads.adwords.axis.v201302.o.LocationSearchParameter;
import com.google.api.ads.adwords.axis.v201302.o.LongAttribute;
import com.google.api.ads.adwords.axis.v201302.o.MoneyAttribute;
import com.google.api.ads.adwords.axis.v201302.o.RelatedToQuerySearchParameter;
import com.google.api.ads.adwords.axis.v201302.o.RequestType;
import com.google.api.ads.adwords.axis.v201302.o.SearchParameter;
import com.google.api.ads.adwords.axis.v201302.o.StringAttribute;
import com.google.api.ads.adwords.axis.v201302.o.TargetingIdea;
import com.google.api.ads.adwords.axis.v201302.o.TargetingIdeaPage;
import com.google.api.ads.adwords.axis.v201302.o.TargetingIdeaSelector;
import com.google.api.ads.adwords.axis.v201302.o.TargetingIdeaServiceInterface;
import com.google.api.ads.adwords.axis.v201302.o.Type_AttributeMapEntry;
import com.google.api.ads.adwords.lib.client.AdWordsSession;
import com.google.api.ads.common.lib.auth.ClientLoginTokens;
import com.google.api.ads.common.lib.conf.ConfigurationLoadException;
import com.google.api.ads.common.lib.exception.ValidationException;
import com.google.api.client.googleapis.auth.clientlogin.ClientLoginResponseException;

public class Adwords {
    public static void main(String[] args) throws ClientLoginResponseException, IOException, ValidationException,
            ConfigurationLoadException {
        String[] locationNames = new String[] { "Paris", "Quebec", "Spain", "Deutschland" };
        String clientLoginToken = new ClientLoginTokens.Builder().forApi(ClientLoginTokens.Api.ADWORDS)
                .fromFile("adwords.properties").build().requestToken();
        AdWordsSession session = new AdWordsSession.Builder().fromFile("adwords.properties")
                .withClientLoginToken(clientLoginToken).build();
        AdWordsServices adWordsServices = new AdWordsServices();
        String[] keywords = getKeywords();

        TargetingIdeaServiceInterface targetingIdeaService = adWordsServices.get(session,
                TargetingIdeaServiceInterface.class);

        TargetingIdeaSelector selector = new TargetingIdeaSelector();

        selector.setRequestType(RequestType.STATS);
        selector.setIdeaType(IdeaType.KEYWORD);

        selector.setRequestedAttributeTypes(new AttributeType[] { AttributeType.KEYWORD_TEXT, AttributeType.SEARCH_VOLUME, AttributeType.AVERAGE_CPC });

        Language language = new Language();
        language.setId(1000L);

        // Countrycodes
        // http://code.google.com/apis/adwords/docs/appendix/countrycodes.html
        Location location = new Location();
        location.setId(2840L);

        RelatedToQuerySearchParameter relatedToQuerySearchParameter = new RelatedToQuerySearchParameter();
        relatedToQuerySearchParameter.setQueries(keywords);

        LocationSearchParameter locationSearchParameter = new LocationSearchParameter();
        locationSearchParameter.setLocations(new Location[] { location });

        LanguageSearchParameter languageSearchParameter = new LanguageSearchParameter();
        languageSearchParameter.setLanguages(new Language[] { language });

        selector.setSearchParameters(new SearchParameter[] { relatedToQuerySearchParameter, locationSearchParameter,
                languageSearchParameter // if not provided locationSearchParameter, languageSearchParameter then result
                                        // is global
        });

        selector.setLocaleCode("US");

        Paging paging = new Paging();
        paging.setStartIndex(0);
        paging.setNumberResults(keywords.length);
        selector.setPaging(paging);

        TargetingIdeaPage page = targetingIdeaService.get(selector);
        if (page.getEntries() != null && page.getEntries().length > 0) {
            for (TargetingIdea targetingIdea : page.getEntries()) {
                Map<AttributeType, Attribute> data = toMap(targetingIdea.getData());
                String kwd = ((StringAttribute) data.get(AttributeType.KEYWORD_TEXT)).getValue();
                Long monthlySearches = ((LongAttribute) data.get(AttributeType.SEARCH_VOLUME)).getValue();
                Money avgCpc = ((MoneyAttribute) data.get(AttributeType.AVERAGE_CPC)).getValue();
                
                System.out.println(kwd + ", " + monthlySearches + ", " + avgCpc.getMicroAmount() / 1000000.0);
            }
        }
    }
    
    public static String[] getKeywords() {
        //Put your keywords here
        return null;
    }
    
    public static Map<AttributeType, Attribute> toMap(Type_AttributeMapEntry[] data) {
        Map<AttributeType, Attribute> result = new HashMap<AttributeType, Attribute>();
        for (Type_AttributeMapEntry entry: data) {
            result.put(entry.getKey(), entry.getValue());
        }
        return result;        
    }
}

Read more...