Debug remote application with Eclipse

When you have remote application with enabled debug it is easy to connect Eclipse IDE and debug the application. Here is the list of steps for it:

1. Open menu Run, select item Debug Configurations.
2. In the left area of the appeared window double click on Remote Java Application
3. Enter all required information about your server. In the screenshot below you can see configurations for the server configured here
4. Click Apply or Debug

Read more...

JBoss remote debugging

As all you well know debugging is very important part of the development process. In this post I will show you how to configure your JBoss AS to allow debugging deployed applications.

We need to configure JBoss's bin/run.conf file. Add the following line at the end:

JAVA_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n $JAVA_OPTS"

This options mean:

• -Xdebug asks JBoss to work with debugging support
• -Xrunjdwp loads JDWP (Java Debug Wire Protocol). This option has its own options:
  1. transport=dt_socket means that sockets will be used for transport
  2. address=8787 means the address(in this case it's the local machine's port 8787) where the socket will be opened
  3. server=y if it is set to 'y' then it means that JBoss will listen for debugger to attach; if it is set to 'n' then it means that JBoss will attach to the debugger at the specified address
  4. suspend=n if it is set to 'y' then it means that JBoss will be launched in the suspended mode and will stay suspended until the debugger is connected

You may also want to check how to debug remote application with Eclipse

Read more...

LDAP Authentication and Search

LDAP is a very widespread way for authentication. In this post I would like to show you how to connect to LDAP server, authenticate user and perform search.

We will use only standard Java classes. So we don't need any dependency. In the example you can see my configurations. Yours may differ (pay attention to Context.SECURITY_PRINCIPAL). Here is the code:

import java.util.Properties;

import javax.naming.Context;
import javax.naming.NamingEnumeration;
import javax.naming.NamingException;
import javax.naming.directory.Attribute;
import javax.naming.directory.Attributes;
import javax.naming.directory.SearchControls;
import javax.naming.directory.SearchResult;
import javax.naming.ldap.InitialLdapContext;

public class LdapTest {
    
    public static void processRequest(InitialLdapContext ctx, String userContext, String filter, String attribute) {
        SearchControls searchControls = new SearchControls();
        searchControls.setSearchScope(SearchControls.SUBTREE_SCOPE);

        NamingEnumeration<SearchResult> searchResults;
        try {
            searchResults = ctx.search(userContext, filter, searchControls);
            
            while (searchResults.hasMoreElements()) {
                SearchResult sr = searchResults.next();
                Attributes attributes = sr.getAttributes();
                Attribute a = attributes.get(attribute);
                if (a != null) {
                    String attrValue = (a.get().toString());
                    System.out.println(attrValue);
                } else {
                    System.out.println("Cannot get data");
                }
            }
        } catch (NamingException e) {
            e.printStackTrace();
        }
    }
        
    private static InitialLdapContext initialiaseLdapContext(String server, int port, String username, String password, String contextDN) {
        boolean initialised = false; 
        Properties properties = new Properties();
        properties.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
        properties.put(Context.PROVIDER_URL, "ldap://" + server + ":"+ port);
        properties.put(Context.SECURITY_AUTHENTICATION, "simple");
        properties.put(Context.SECURITY_PRINCIPAL, "uid=" + username + "," + contextDN); //YOU MAY NEED TO CHANGE CODE HERE
        properties.put(Context.SECURITY_CREDENTIALS, password);

        InitialLdapContext ctx = null;
        try {
            // Create initial context
            ctx = new InitialLdapContext(properties, null);
            initialised = true;
        } catch (NamingException e) {
            initialised = false;
        } finally {
            if (initialised) {
                System.out.println("Initialization success");
            } else {
                System.out.println("Initialization fail");
            }
        }
        return ctx;
    }

    public static void main(String[] args) {
        String contextDN = "dc=test,dc=com";
        InitialLdapContext ctx = initialiaseLdapContext("localhost", 389, "test@test.com", "test" , contextDN);
        try {
            if (ctx != null) {
                processRequest(ctx, contextDN, "(uid=qwertTest@test.com)", "street");
                ctx.close();
            }
        } catch (NamingException e) {
            e.printStackTrace();
        }
    }
}

Read more...

Fast bulk loading to the database

Whenever you have to insert a big amount of data consider using not usual 'Insert Into', but the load command. For MySQL and Oracle it's 'Load Data Infile'. Here is the small example how to import usual CSV file with the header to MySQL:

load data local infile 'load.csv'
into table someTable
character set utf8 
fields terminated by ',' 
optionally enclosed by '"'
lines terminated by '\n'
ignore 1 lines

For more information view official documentation

Read more...

Fast way to kill many MySQL processes

I think everyone who works for some time with MySQL has been in trouble with suspended processes or reaching the limit of connections. The fastest way to fix it is to restart MySQL service

service mysql restart

But if it's the production server and you cannot afford restarting the service then here is the tip for you.

At first we need to find out what processes are causing troubles. Let's login to MySQL:

mysql -uroot -ptoor

And watch the list of all processes

select * from information_schema.processlist;

or

show processlist;

Let's assume we need to delete processes that are active more then 1000 seconds. To watch them we need this query:

select * from information_schema.processlist where TIME > 1000;

Let's format the output and save it to the file (if you have problems saving to the file then check this post)

select concat('kill ',id,';') 
into outfile '/home/anton/mysql/kill.sql' 
from information_schema.processlist 
where TIME > 1000;

After this let's load the created file

source /home/anton/mysql/kill.sql;

Thus we have deleted multiple MySQL processes with our own criteria.

Read more...

Reading XML files using XPath

Nowadays declarative programming has took a well-deserved place in imperative languages. In Java we can declare what we want in annotations or in XML. The main plus of XML is that we can reconfigure our application in one place and update it without rebuilding.

There a lot of ways to read data from XML. The most common ones are: XPath, XSLT and XQuery.

• XPath consists of path expressions, conditions and some XPath functions
• XSLT(Extensible Stylesheet Language (with) Transformation) consists of XPath and some transformations
• XQuery consists of XPath and a Query Language. This query language is pretty powerful but it does not have strong underlying mathematics like SQL.

In this tutorial we will develop a simple XPath XML file parser. But as you probably know, there are different kind of parsers:

• DOM(Document Object Model) parsers. DOM API creates a DOM tree in memory for a XML document
• SAX(Simple API For XML) parsers. They are event driven(as they parse documents they invoke the callback methods)

We will develop DOM parser. For this purpose we will use only standard Java classes without any additional dependencies.
As an example XML file we will use JBoss 5.1 datasource file. Here it is:

<?xml version="1.0" encoding="UTF-8"?>

<datasources>
    <xa-datasource>
        <jndi-name>DevDbDS</jndi-name>
 <!--
        <use-java-context>false</use-java-context>
 -->
 <xa-datasource-class>
  com.mysql.jdbc.jdbc2.optional.MysqlXADataSource
 </xa-datasource-class>
 <xa-datasource-property name="URL">
  URL:jdbc:mysql://localhost:3306/dev_db
 </xa-datasource-property>

        <user-name>root</user-name>
        <password>root</password>
 <!--
        <security-domain>VocdmsDSEncryptedLogon</security-domain>
 -->
    <xa-datasource-property name="characterEncoding">UTF-8</xa-datasource-property>

 <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter</exception-sorter-class-name>
        <min-pool-size>5</min-pool-size>
        <max-pool-size>20</max-pool-size>
        <blocking-timeout-millis>30000</blocking-timeout-millis>
        <idle-timeout-minutes>15</idle-timeout-minutes>
        <prefill>true</prefill>
        <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
        <connection-property name="characterEncoding">UTF-8</connection-property>
        <connection-property name="autoReconnect">true</connection-property>
        <connection-property name="maxReconnects">4</connection-property>
        <connection-property name="initialTimeout">3</connection-property>
        <metadata>
            <type-mapping>mySQL</type-mapping>
        </metadata>
    </xa-datasource>
</datasources>

Next we need to remind ourselves how to write XPath expressions. You can do it here

Now finally we are ready to write the code to get some information from the datasource. In this example let's assume we need user name, password and URL(you may need this data to establish java.sql.Connection). So here is our code:

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathFactory;

import org.w3c.dom.Document;
import java.io.*;

public class XmlParser {
 //Immutable object for data storage
 private static class ConnectionInfo {
  private final String url;
  private final String userName;
  private final String password;
  
  public ConnectionInfo(String url, String userName, String password) {
   this.url = url;
   this.userName = userName;
   this.password = password;
  }
 
  public String getUrl() {
   return url;
  }
  public String getUserName() {
   return userName;
  }
  public String getPassword() {
   return password;
  } 
  @Override
  public String toString() {
   return "URL: " + url +
    "\nuserName: " + userName + 
    "\npassword: " + password;
  }
 }

 public static void main(String[] args) {
  System.out.println(parseDataSource("mysql-ds.xml"));
 }

 public static ConnectionInfo parseDataSource(String fileName) {
  try {   
      DocumentBuilderFactory domFactory = DocumentBuilderFactory.newInstance();
      domFactory.setNamespaceAware(true); // never forget this!
      DocumentBuilder builder = domFactory.newDocumentBuilder();
      Document doc = builder.parse(new FileInputStream(fileName));

      XPathFactory factory = XPathFactory.newInstance();
      XPath xpath = factory.newXPath();
      String url = xpath.evaluate("//datasources/xa-datasource/xa-datasource-property[@name='URL']/text()", doc).trim();     
      String userName = xpath.evaluate("//datasources/xa-datasource/user-name/text()", doc).trim();
      String password = xpath.evaluate("//datasources/xa-datasource/password/text()", doc).trim();
      return new ConnectionInfo(url, userName, password);
  } catch (Exception ex) {
      ex.printStackTrace(); 
      return null;
  }
     } 
}

Read more...

Google Analytics API usage example

It's very useful in SEO business to keep track of different user's action on your site. One of the best tools for such kind of analysis is Google Analytics. Furthermore it's free and provides the API. In this post I will show you the simple example of usage of this API.

As usual let's start with the Maven dependencies. But unfortunately Google do not provide jars that we need in any repository. Thus we need to download and install them to Maven by ourselves. Google for "gdata core jar" and "gdata analytics jar". When you will have those files execute in command line following code to install these jars:

mvn install:install-file -Dfile=gdata-core-1.0.jar -DgroupId=com.google.gdata -DartifactId=gdata-core -Dversion=1.0 -Dpackaging=jar -DgeneratePom=true
mvn install:install-file -Dfile=gdata-analytics-2.1.jar -DgroupId=com.google.gdata -DartifactId=gdata-analytics -Dversion=2.1 -Dpackaging=jar -DgeneratePom=true

Add to pom.xml following Maven dependencies:

<dependency>
    <groupId>com.google.gdata</groupId>
    <artifactId>gdata-core</artifactId>
    <version>1.0</version>
</dependency>
<dependency>
    <groupId>com.google.gdata</groupId>
    <artifactId>gdata-analytics</artifactId>
    <version>2.1</version>
</dependency>

Also you may need Guava(which is Google's library with different basic and universal functions.

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>10.0.1</version>
</dependency>

After maven fetches us our new dependencies we may continue and develop Java source code for Google Analytics API data fetching.

public static DataFeed getDataFeed(String dimensions, String metrics, String userName, String password, 
        String tableId, String startDate, String endDate, int startIndex) throws IOException, ServiceException  {
    System.out.println("Building data feed");
    // Configure GA API.
    AnalyticsService as = new AnalyticsService("My application name");

    // Client Login Authorization.
    as.setUserCredentials(userName, password);
    DataQuery query = new DataQuery(new URL("https://www.google.com/analytics/feeds/data"));
    query.setIds(tableId);
    query.setDimensions(dimensions);
    query.setMetrics(metrics);
    query.setSort("-" + metrics);
    query.setMaxResults(10000);        
    query.setStartDate(startDate);
    query.setEndDate(endDate);
    query.setStartIndex(startIndex);
    URL url = query.getUrl();
    
    System.out.println("URL: " + url.toString());

    // Send our request to the Analytics API and wait for the results to
    // come back.
    DataFeed feed = as.getFeed(url, DataFeed.class);
    return feed;
}

As a parameters you should provide

• dimensions and metrics (you can get both from Google Analytics API Reference)
• Google Analytics credentials (user name and password)
• Google Analytics Profile ID(also called table ID)
• date range for quering
• start index(We need to use start index because Google Analytics service won't return result set with more then 10000 records)

Let's save data that we got to CSV file using opencsv:

public static void saveAnalyticsFile(DataFeed feed, String fileName) {
 System.out.println("Saving to file");
    try {
        CSVWriter writer = new CSVWriter(new FileWriter(fileName), ',' ,CSVWriter.DEFAULT_QUOTE_CHARACTER, '\\');
        
        DataEntry singleEntry = feed.getEntries().get(0);
        List<String> feedDataNames = new ArrayList<String>();
        List<String> dataRowValues = new ArrayList<String>(LoaderConstants.GA_FILE_HEADER.length);

        // Put all the dimension and metric names into an array.
        for (Dimension dimension : singleEntry.getDimensions()) {
            feedDataNames.add(dimension.getName());
        }
        for (Metric metric : singleEntry.getMetrics()) {
            feedDataNames.add(metric.getName());
        }

        //write header
        writer.writeNext(LoaderConstants.GA_FILE_HEADER);

        for (DataEntry entry : feed.getEntries()) {         
         //assuming that the first entry is a keyword and others are numeric values
            if (!entry.stringValueOf(feedDataNames.get(0)).equals("(not provided)") && 
                    !entry.stringValueOf(feedDataNames.get(0)).equals("(not set)")) {
             dataRowValues.add(entry.stringValueOf(feedDataNames.get(0))); //keyword
             for (int i = 1; i < feedDataNames.size(); i++) {
                 Double d = Double.parseDouble(entry.stringValueOf(feedDataNames.get(i)));
                 String googleVal = new Long(d.longValue()).toString();
                 dataRowValues.add(googleVal); 
             }
                writer.writeNext(dataRowValues.toArray(new String[0]));
             
                dataRowValues.clear();
            }
        }
        writer.close();        
        System.out.println("Saving is done");
        
    } catch (Exception e) {
        System.out.println("Cannot save file ");
        throw new RuntimeException(e);
    }
}

And finally here is the main method:

public static void main(String[] args) {
 int startIndex = 1;
 boolean fetch = true;
 while (fetch) {
     DataFeed dataFeed = getDataFeed("ga:keyword", "ga:visits", "userName", "password", 
                "profileId", "2012-08-13", "2012-08-19", startIndex);
        CsvWriteUtils.saveAnalyticsFile(dataFeed, "test");
        startIndex += 10000; //Google Analytics Max result value
        //continue fetching until we receive all data
        if (startIndex > dataFeed.getTotalResults()) {
         fetch = false;
        }
 }
}

Read more...