Accessing Solr Cloud on AWS from SolrJ

We have a Solr cloud installation on AWS EC2 instances. We use the SolrJ Client from our Java application. Till date we used to have a Solr Cloud installation on our local machine in order to test the code against. As the team started growing, we realized that we should have a way to access the AWS Solr Cloud from our local boxes, so that the Solr Cloud setup on local is not a blocker for feature development.

When I started looking around the web for a solution to this issue, I had to mix a few things from a few different documentation pages and this stackoverflow page. Decided to write this blog post to help out others who are running into the same issue. I hope it helps.

So first let's try to understand the reason behind why you get the java.net.UnknownHostException from the SolrJ client.
When the solr cloud is run, each instance registers itself with the zookeeper that is running. This is done with the private IP of the solr machine. So when SolrJ connects to zookeeper, it gets the private IP. And then SolrJ tries to hit the Solr instance with the private IP, leading to an unknown host exception.

In order to get around this you can follow these steps:
Step 1: Run the solr instance with a host name. Zookeeper will use this hostname.

./bin/solr start -cloud -h hostname -p 8985 -z localhost:2181

Step 2: Add the host entry to the /etc/hosts file on your local machine corresponding to the public ip of the solr machine.

And you are done. You can run the java application that uses SolrJ client on your local machine and access the Solr Cloud running on your AWS instance :)

Managing Multiple Github Accounts from a single machine

Recently, I ran into a situation where I needed to manage multiple Github accounts from a single computer. I have two github account, one related to work and the other for my personal repositories.
It is a very simple combination of ssh and git config. Before I start explaining the steps for the same, let me clarify that the following steps are meant for LINUX/UNIX users.

1. Set up SSH keys

You will need to set up two different ssh keys for the two accounts.

$ ssh-keygen -t rsa -b 4096 -C "work.emailid"
Generating public/private rsa key pair.
Enter file in which to save the key (/home/****/.ssh/id_rsa):       
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/anjana/.ssh/id_rsa.
Your public key has been saved in /home/anjana/.ssh/id_rsa.pub.

$ssh-keygen -t rsa -b 4096 -C "personal.emailid"
Generating public/private rsa key pair.
Enter file in which to save the key (/home/****/.ssh/id_rsa): /home/****/.ssh/id_rsa_personal       
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/anjana/.ssh/id_rsa_personal.
Your public key has been saved in /home/anjana/.ssh/id_rsa_personal.pub.

Make sure that when prompted for the filename when generating the ssh keys for the second account, you give the appropriate filename.

2. Add the ssh key to your github account

> On your github account, Go to your Account Settings
> Click "SSH Keys" then "Add SSH key"
> Paste your key into the "Key" field and add a relevant title
> Click "Add key" then enter your Github password to confirm
> Repeat this step for your other account with the appropriate keys.

3. Create a ssh configuration file to manage the two separate keys

$ touch ~/.ssh/config

Add the following in your ~/.ssh/config file.
# Default GitHub
Host personal-github.com
HostName github.com
User git
PreferredAuthentications publickey
IdentityFile ~/.ssh/id_rsa_personal

# Work GitHub
Host github.com
HostName github.com
User git
PreferredAuthentications publickey
IdentityFile ~/.ssh/id_rsa

Once you save this file, you need to configure your git repos on local accordingly.

4. Setup your repo

Replace "git@github.com" part in the remote origin of all your personal repositories to "git@personal-github.com"
And that's it. From now on, the appropriate user account will be used for pushing to these repositories.

Interfaces in Java8

Interfaces are a key feature of object oriented programming. An interface provides a set of methods that an implementing class must provide. One can assign instances of the class to variables of the interface type. As of Java8, an interface can contain default methods that an implementing class can inherit or override. Default methods (also known as Defender Methods) enable us to add new functionalities to interfaces without breaking the classes that implements the interface.

1. Declaring an interface

Interfaces are declared by specifying a keyword "interface"

2. Implementing an interface

The class implementing the interface has to provide the implementation of all the methods in the interface


The @Override annotation tells the compiler that this method is inherited from the interface.

Output

3. Default methods in Interface

Java 8 onwards, you can provide default implementation of methods in interfaces. If the class implementing the interface does not provide an implementation of the method, then the default implementation is used.


Let's change the main() method to call the newMethod. Observe that the interface implementation has not changed.

4. Using Default Methods or Abstract Classes

After the introduction of default methods, it may seem that there are no differences between abstract classes, and interfaces. However, it is not so. Abstract classes can define constructor. They can have a state associated with them. In contrast, default methods can only be implemented in terms of invoking other interface methods, with no reference to a particular implementation's state.

5. Default methods and Multiple inheritance ambiguity

A Java class can implement multiple interfaces. Each interface can define default method with same method signature, therefore, the methods can conflict with each other.

Let's consider an example:
Declare two interfaces with default methods implemented with the same method signature.




The above code will give a compilation error:
java: class InterfaceImpl inherits unrelated defaults for method() from types InterfaceA and InterfaceB

We need to provide an implementation of the default method in order to resolve this conflict.

Summary

To summarise, default methods enable addition of new functionality without breaking existing implementations.
When we extend an interface containing the default method, we can do one of the following:
1. Not override the default method and the implementing class will inherit the default method.
2. Override the default method similar to other methods we override in subclass.
3. Redeclare default method as abstract, which forces the subclass to override it.

Reference: Java Documentation and D Zone

Database Indexing Basics

Database Indexing allows us to cut down the number of rows/records that need to be examined when a select query with a where clause is executed.
Let's try to understand what happens when a select query is executed on a database table without an index.
Suppose we have an Employee table, with fields 'Employee_Id' and 'Employee_Name'. We want to execute the following select query on this table:


When this query is executed, what goes on behind the scenes? Every single row is checked to see if the employee_name matches with 'abc'. Which effectively means that the entire table will have to be scanned. This is known as full table scan.

An index helps us avoid a full table scan. It is a data structure, that stores the values for a certain specific column of a table. Let's look at the common data structures that are used for database indexing.

Data Structures for Indexing

1. B-trees
B-trees are the most commonly used data structures for indexes. As they are time-efficient for lookups, deletions and insertions. All these operations can be done in logarithmic time. Also, the data being stored inside B-tree can be sorted.

2. Hash Tables
The indexes that use hash tables are generally referred to as hash index. Hash tables are extremely efficient for looking up values. So queries that look for an exact match can be executed very fast. The way a hash index works is that the key is the column value, and the value in the hash table is a pointer to the row data in the table. However, hash tables are not sorted data structures. And hence, for other type of queries, they may not be efficient.

3. R-tree
This is commonly used with spatial databases. It helps with queries like "find all the coffee shops within 2 kilometers of my location".

4. Bitmap Index
Works for columns that have many instances of those values, that is columns with low selectivity. For example, a column with boolean values.

So, what is the cost of having a database index?
The first thing is the index takes additional space, and the larger your table, the larger is your index. Every time you perform an add, delete or update operation, the same operation needs to be performed on the index as well.

Creating an index
The following snippet shows how to create an index on a single column and multiple columns.


As a general rule, an index should only be created on a table if the data in the indexed column will be queried frequently.

References:
http://www.programmerinterview.com/index.php/database-sql/what-is-an-index/

Immutability in Java - What it takes to build an immutable class

Immutability means something that cannot be changed. In java an immutable class is one whose state cannot be changed once it has been created. This post aims to give a guideline on how to make a class immutable. The post contains an example of creating an immutable class.
The complete code is available on Github

The Java documentation gives a list of guidelines for creating an immutable class. We will try to understand it better.

To make a class immutable, follow the following steps:

1. Declare the class as final. 
2. Make all its fields final and private.
3. For all mutable fields, make sure that the class creates a copy and only returns the copy to the calling code.
4. Do not provide any setter methods.

Let's try to understand why we need to do all of the above for making a class immutable. We need to ensure that the subclasses do not override any of the class methods. Declaring the class as final ensures that the class cannot be overridden. A more sophisticated approach is to make the constructor private and use a factory method to create an instance of the class. Making all the fields private will ensure that the fields cannot be changed outside the class, and making them final will ensure that we do not alter the field even by mistake. For all the mutable fields, we are making sure that no other object from outside can change the data by creating a defensive copy of the object and only returning this copy to the calling code. Setter methods are usually used to change the state of the object and as the goal of an immutable class is to avoid state changes, hence we do not provide any setter methods.

Lets look at an example of Immutable class:



We have three fields in the class, first field of Integer type, second field of String type, and third field of Date type. The String and Integer classes are immutable, however the Date class is mutable. Thus, we create a new Date object when assigning the Date to the class field.
That's all for immutability.


Reference: Java Documentation

Getting Started with TestNG

TestNG is a testing framework which, inspired by JUnit and NUnit, but introduces things like dependency testing, grouping concept to make testing more powerful and easier to do. It is designed to cover all categories of tests: unit, functional, end-to-end, integration, etc...
The NG in TestNG stands for Next Generation. TestNG requires JDK7 or higher.
Reference: http://testng.org/doc/index.html

In this post, we will be discussing how to use TestNG to write a simple test:

Firstly, add the TestNG library in the pom.xml


Lets create a simple Java class that has a method that returns the String passed to it.


Create a test case like this:


Thats it, a simple TestNG test case is created.

Setting up Mac OSx for Foundation5

Foundation is an open source responsive front end framework maintained by zurb.com. It has a seemingly endless list of features that allows you to build responsive web applications within no time. In this post, we will set up our Mac OSx for developing foundation projects.

For foundation you need the following packages to be installed on your system. The following steps assumes the presence of homebrew on your system. If you do not have homebrew installed, please install it. The instructions are available on homebrew website.

Git

Git is version control system that basically commands the tools that will be used for command line in Github.

$ brew doctor
$ brew update
$ brew install git

NodeJs

NodeJs allows Javascript to run outside the Browser, and is required by Bower to package all the files in the project and link them together.

$ brew doctor
$ brew update
$ brew install node

Ruby 1.9+

The script used by Foundation to download all Sass files and scaffolding the project is written in Ruby.

$ brew doctor
$ brew update
$ brew install ruby

Installing the foundation CLI

$ gem install foundation
$ npm install -g bower grunt-cli

g stands for global

That's it, you have got foundation installed on your system. Now let's create a new foundation project.

Creating a new Foundation project

$ foundation new myproject
Creating ./myproject
      create  myproject
Cloning into 'myproject'...
remote: Counting objects: 120, done.
remote: Total 120 (delta 0), reused 0 (delta 0), pack-reused 120
Receiving objects: 100% (120/120), 40.77 KiB | 0 bytes/s, done.
Resolving deltas: 100% (48/48), done.
Checking connectivity... done.
Installing dependencies with bower...
./myproject was created

That's all that has to be done for you to start building foundation 5 projects.

Compiling Sass into CSS

This section takes you through the steps for creating a Sass project. Sass stands for Sassy CSS. It builds on the existing CSS syntax.

$gem install compass

After this you can create a new Foundation Sass project.

Reference : Zurb Foundation Documentation