Sublime snippet for creating getter and setter methods in Java class.

1. Go to 'Tools' -> 'New Snippet'
2. Replace the default content with the below

   <snippet>
    <content><![CDATA[ public ${1:returnType} get${2:Property}(){
     return ${3:property};
    }
   
    public void set${2:Property}(${1:returnType} ${3:property}){
     this.${3:property} = ${3:property};
    }
   ]]></content>
    <tabTrigger>gs</tabTrigger> 
   </snippet>
   

3. Save the file in sublime-text-2/Packages/User/urfilename.sublime-snippet
4. Open the java file that needs getter and setter methods
5. Type "gs" and hit 'tab'
6. Snippet should be created at this point and returnType should be highlighted. Type in the returnType of the property for which these getter and setters are being created
7. Hit 'tab'. Property will be highlighted. Type in the property name with first letter in uppercase.
8. Hit 'tab'. type in the property name as it is

Load a Resource from Classpath in Groovy

Lets look at how we can load a resource from classpath in groovy when groovy is used as a part of java application. Put you resource beside the groovy class and do below

ReflectionUtils.getCallingClass(0).getResourceAsStream("testGfile.txt")

Write Nutch Plugin Code in Groovy

In this post, let us look at how we can integrate groovy into nutch code base so that one can write plugin code in groovy.

Below are the steps

1.Edit "src/plugin/myplugin/ivy.xml" of your plugin to include groovy jar. After edit, dependencies section of the file should look like below

<dependencies>
 <dependency org="org.codehaus.groovy" name="groovy-all" rev="1.7.4"/>
</dependencies>

2.Edit "src/plugin/myplugin/plugin.xml" to include groovy jar. Runtime section of the file after edit should look like below

<runtime>
      <library name="my-plugin.jar">
         <export name="*"/>
      </library>
      <library name="groovy-all-1.7.4.jar"/>
   </runtime>

3.Edit "ivy/ivy.xml" to include below in the dependencies section.

<dependency org="org.codehaus.groovy" name="groovy-all" rev="1.7.4"/>

4.Edit "src/plugin/build-plugin.xml" and add the below taskdef

<taskdef name="groovyc"
 classname="org.codehaus.groovy.ant.Groovyc">
    <classpath refid="classpath"/>
  </taskdef>

add the below target

<target name="groovyCompile">
   <echo message="Compiling groovy classes in plugin: ${name}"/>
   <groovyc
     srcdir="${src.dir}"
     includes="**/*.groovy"
     destdir="${build.classes}">
      <classpath refid="classpath"/>
   </groovyc>
  </target>

and modify target named "compile" to depend on "groovyCompile"

change below
<target name="compile" depends="init,deps-jar, resolve-default">


to
<target name="compile" depends="init,deps-jar, resolve-default, groovyCompile">

That's it. Now you should be able to add groovy classes to your plugin and used them in java classes. Also, if you are working in eclipse, do not forget to add groovy-all-1.7.4.jar to your classpath.

Creating Nutch Distribution with Custom Plugin Code and Running

In this post, we are going to talk about how we can build nutch with custom plugin code. Look here to see how we can create our custom plugin. After the plugin development is done, to make a distribution, we need to do below

1. Go to nutch project folder (lets assume it is "~/workspaces/nutch")
2. Run "ant tar"
3. The above command creates "~/workspaces/nutch/dist" (here you can find the distribution nutch-1.3.tar.gz) 
4. nutch-1.3.tar.gz is also unzipped into the folder "~/workspaces/nutch/dist/nutch-1.3" 
5. change directory to "~/workspaces/nutch/dist/nutch1.3/runtime/local"
6. verify that folder "urls" exists in this directory. If not, create it. Add some seed  urls to it.
7. While developing, "plugin.folders" property in "conf/nutch-site.xml" has a value of "./src/plugins". This does not work when you are working with distribution. Change the value of this property to wherever plugin jars are located. By default, these jars are in "~/workspaces/nutch/dist/nutch1.3/runtime/local/plugins" folder. Since we are already in folder "local", change the value of "plugin.folders"  property value in "conf/nutch-site.xml" to "./plugins".  If you for got to do this, you might see an error: Caused by: java.lang.RuntimeException: x point org.apache.nutch.net.URLNormalizer not found.  at org.apache.nutch.net.URLNormalizers.(URLNormalizers.java:122)at org.apache.nutch.crawl.Injector$InjectMapper.configure(Injector.java:70)
8. Now run nutch using command like "bin/nutch crawl urls -dir crawl -depth 2 -topN 4". A good explanation of what everything means in this command can be found here.      

That's it... 

Java Regular Expressions Test Harness

Regular Expressions are best understood when we try them using java program with different inputs and analyzing output while reading the definitions.

Java Regex page provides a class that we can use to try out regular expressions with different inputs. But, the issue with this program is, it does not work in eclipse because of the access to Console from eclipse. If you are experiencing that issue, the below program can be used to try them out.

package org.apache.nutch.util;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class RegexTestHarness {
 public static void main(String[] args){
  try {
         BufferedReader console = new BufferedReader(new InputStreamReader(System.in));
         System.out.println("Enter regual expression: ");
         String regex;
   regex = console.readLine();
         while (!regex.equalsIgnoreCase("exit")) {
 
             Pattern pattern = Pattern.compile(regex);
 
             System.out.println("Enter input string to search: ");
             Matcher matcher = pattern.matcher(console.readLine());
 
             boolean found = false;
             while (matcher.find()) {
                 System.out.println("I found the text '"+ matcher.group()+ "' starting at index " +matcher.start()+ " and ending at index "  +matcher.end());
                 found = true;
             }
             if(!found){
              System.out.println("No match found.%n");
             }
          System.out.println("Enter regual expression: ");
    regex = console.readLine();
         }
  } catch (IOException e) {
   e.printStackTrace();
  }
    }
}

Parse String to Java Date and Format Date to Localized String

In this post, Let's look into how we can convert a String to Date object in Java and vice versa.

Solid understanding of Date class is essential to understand this post or anything related to java dates. I have written a post explaining the basics of Date class.

I have a string "october 21, 2011 19:08". I would like to convert this to Date object. How can I do this? Also, how can I convert that Date back to localized String?

First, Lets look at the code.

1:  import java.text.ParseException;  
2:  import java.text.SimpleDateFormat;  
3:  import java.util.Date;  
4:  import java.util.TimeZone;  
5:    
6:  public class DateTest {  
7:    
8:       public static void main(String[] args) {  
9:            try {  
10:                 String dateStr = "october 21, 2011 19:08";  
11:                 String pattern = "MMMM dd, yyyy HH:mm";  
12:                 TimeZone timezoneOfDateStr = TimeZone.getTimeZone("Asia/Kolkata");  
13:                   
14:                 SimpleDateFormat sd = new SimpleDateFormat(pattern);  
15:                 sd.setTimeZone(timezoneOfDateStr);  
16:                 Date date = sd.parse(dateStr);  
17:                   
18:                 System.out.println("Converting : ");  
19:                 System.out.println("     Date String: "+dateStr);  
20:                 System.out.println("     pattern: "+pattern);  
21:                 System.out.println("     time zone: Asia/kolkata");  
22:                 System.out.println();  
23:                 System.out.println("Converted.");  
24:                 System.out.println("System Stored the above Date as : "+date.getTime());  
25:                 System.out.println();  
26:                 String patternWithTimezone = "MMMM dd, yyyy HH:mm zzzz";  
27:                 SimpleDateFormat sdWithTZ = new SimpleDateFormat(patternWithTimezone);  
28:                 sdWithTZ.setTimeZone(timezoneOfDateStr);  
29:                 System.out.println("Formatting the date back to human readable format in the same timezone");  
30:                 System.out.println("     here it is: "+sdWithTZ.format(date));  
31:                   
32:                 System.out.println();  
33:                 sdWithTZ.setTimeZone(TimeZone.getTimeZone("GMT"));  
34:                 System.out.println("Formatting the date to human readable format in GMT");  
35:                 System.out.println("     here it is: "+sdWithTZ.format(date));  
36:            } catch (ParseException e) {  
37:                 e.printStackTrace();  
38:            }  
39:       }  
40:    
41:  }  
42:    

Output

Converting : 
 Date String: october 21, 2011 19:08
 pattern: MMMM dd, yyyy HH:mm
 time zone: Asia/kolkata

Converted.
System Stored the above Date as : 1319204280000

Formatting the date back to human readable format
 here it is: October 21, 2011 19:08 India Standard Time

Formatting the date to human readable format in GMT
 here it is: October 21, 2011 13:38 Greenwich Mean Time

Explanation

1. Line 10: The date that needs to be converted
2. Line 11: Pattern of the date that is defined in line 10. To understand how to build this pattern, read the table in the java doc of SimpleDateFormat class which explains about the letters and how they are interpreted (example: M is interpreted as month, y as year, d as Date, H as hour, m as min, z as timezone etc)
3. Line 12: Timezone of the above date. Why timezone? A date by itself does not tell the whole story. (example: Oct 21, 2011 08:00 in newyork is same as Oct 21, 2011 11:00 in Las Angeles). So, just saying Oct 21, 2011 would not tell the whole story, we also need to tell the system what timezone that time belongs to. In this case, we are saying that the date belongs to "Asia/Kolkata timezone" / "India Standard Time".
4. Lines 14,15,16: Create SimpleDateformat class. Tell it what pattern the string is in, What timezone it belongs to. Parse the date string and get date object.
5. Lines 18 to 25: Print some details. It also prints the date in milliseconds. Why in milliseconds? Read Java Date Explanation
6. Lines 26 to 30: Format the date to human readable format in India Standard Time and print.
7. Lines 32 to 35: Format the date to human readable format in GMT and print.

Java Date Explanation

In this post, let's look into some basics about Date class which creates lots of confusion not only among beginners but even among experienced programmers. It is understandable given that even sun folks couldn't understand it correctly when they initially wrote the Date class. That is the reason the Date class has below explanation in it's java class.

Prior to JDK 1.1, the class Date had two additional functions. It allowed the interpretation of dates as year, month, day, hour, minute, and second values. It also allowed the formatting and parsing of date strings. Unfortunately, the API for these functions was not amenable to internationalization.

What I am going to discuss below is very important. It will make your understanding of dates whole lot easier. Pay special attention.

When you think of Date, think of milliseconds from epoch (January 1, 1970) . Date in java is just a wrapper around milliseconds. Let me clarify this with an example.

1:  import java.util.Date;  
2:  import java.util.TimeZone;  
3:    
4:  public class DateTest {  
5:    
6:       public static void main(String[] args) {  
7:            TimeZone.setDefault(TimeZone.getTimeZone("GMT-5"));  
8:            Date gmtMinusFiveDate = new Date();  
9:              
10:            TimeZone.setDefault(TimeZone.getTimeZone("GMT+1"));  
11:            Date gmtPlusOneDate = new Date();  
12:              
13:            TimeZone.setDefault(TimeZone.getTimeZone("GMT+7"));  
14:            Date gmtPlusSevenDate = new Date();  
15:              
16:            System.out.println("GMT-5 date milliseconds: "+gmtMinusFiveDate.getTime());  
17:            System.out.println("GMT+1 date milliseconds: "+gmtPlusOneDate.getTime());  
18:            System.out.println("GMT+7 date milliseconds: "+gmtPlusSevenDate.getTime());  
19:     }  
20:  }  

Output

GMT-5 date milliseconds: 1319391112399
GMT+1 date milliseconds: 1319391112400
GMT+7 date milliseconds: 1319391112400

Above we created Date objects in different time zones (GMT-5, GMT+1, and GMT+7) and printed their millisecond value.

Surprised that all the date instances print same value for milliseconds? (the 1 millisecond difference between 1st and 2nd line in the output is because of when in the program the date object is created). Shouldn't they be hours apart? Not really. This is how it works.

Let's say, we created a Date instance in a machine running in Arizona Timezone (GMT-7) on Oct 21, 2011 at 06:00. Think of it as below is what happens (although the machine would always have millis calculated in GMT from Jan 1, 1970 midnight)

• Convert the time to GMT. So, Oct 21, 2011 06:00 will be converted to Oct 21, 2011 13:00 (remember GMT-7, so just add 7 hours)
• Calculate number of milliseconds elapsed from Jan 1, 1970 to Oct 21, 2011 13:00 (lets say )
• Create a Date object that is a wrapper around this milliseconds value.

that is the reason, regardless of where the machine is running (or what the default timezone is), the date object would only represent the number of milliseconds since January 1, 1970, 00:00:00 GMT. That's it. That is the reason all three lines in the output show the same value.

Solid understanding of the above concept is essential when one is dealing with dates in Java. If you did not understand the above, go back again and read one more time. Once you understand the above concept everything else should be easy to understand.

See my post about Parse String to Java Date and vice versa