SSL, tomcat and self signed certificates

I have been through the process of getting an application to work via https before but I have always muddled through and never documented exactly what I have done.

I must also admit that I never quite understood exactly what I had done and why.

One problem is that I can not find one place that covers everything you need in one place.

Now I think I am pretty well there.

This post describes what I have done to get a simple client application working using HTTPS posting to tomcat server and using a self signed certificate with the help of several refereneces in particular:

1. How to create a self signed certificate
2. Tomcat SSL Configuration How-To
3. Developing Web Services with Apache Axis by Tong Ka Iok, Kent ISBN 1-4116-7032-9

The last reference has a chapter which is pretty well SSL for dummies and explains things very nicely.

The steps that need to be done are:

1. Create a self signed certificate authority (CA)
2. Sign a test key via the CA
3. Add both these keys to a keystore
4. Setup the application (client) and tomcat (the server) to use this keystore.

1) Create a self signed certificate authority (CA) and keystore

This is described in How to create a self signed certificate, but I will show the steps here

What is happening here:

you will create a CA that later will be added to your keystore file. By adding this CA to your keystore you are saying it is trusted like verisign and any certificates signed by it are then also trusted.

1.1) make a directory to hold the certs and keystore. This might be something like:

C:\ssl

1.2) generate a private key for the server

openssl genrsa -des3 -out server.key 1024

1.3) generate a CSR (Certificate Signing Request)

openssl req -new -key server.key -out server.csr

1.4) Remove the passphrasse from the key

cp server.key server.key.org

openssl rsa -in server.key.org -out server.key

1.5) Generate the self signed certificate

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

2) Create a certificate for tomcat and add both to the keystore

2.1) cd to where the keystore is held. This might be something like:

C:\ssl

2.2) Create a keypair for 'tomcat'

keytool -genkey -alias tomcat  -keyalg RSA -keystore tomcat.ks

2.3) Generate a CSR (Certificate Signing Request) for tomcat

keytool -keystore tomcat.ks -alias tomcat -certreq -file tomcat.csr

2.4) create unique serial number

echo 02 > serial.txt

2.5) Sign the tomcat CSR

openssl x509 -CA server.crt -CAkey server.key -CAserial serial.txt -req -in tomcat.csr -out tomcat.cer -days 365

2.6) Import the server CA certificate into the keystore

keytool -import -alias serverCA -file server.crt -keystore tomcat.ks

2.7) add the tomcat certificate to the keystore

keytool -import -alias tomcat -file tomcat.cer -keystore tomcat.ks

3) Tomcat configuration

3.1) Tomcat needs to be configured to use SSL

This is described in more detail at Tomcat SSL Configuration How-To

However all that is needed here is to edit the server.xml to enable SSL

This section is already in the server.xml but commented out.

NB that the location of the keystore has been added.

<!-- Define a SSL HTTP/1.1 Connector on port 8443

This connector uses the JSSE configuration, when using APR, the

connector should be using the OpenSSL style configuration

described in the APR documentation -->

<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"

maxThreads="150" scheme="https" secure="true"

keystoreFile="C:\ssl\tomcat.ks"

keystorePass="changeit"

clientAuth="false" sslProtocol="TLS" />

3.2) test tomcat

start tomcat and go to https://localhost:8443/

your browser will return an error such as "sites certificate is not trusted"

3.3) import the CA certificate server.crt into your browser's tructed root certificates

3.4) test again at https://localhost:8443/

this time you should see the tomcat home page

4) Test your application

4.1) I have a unit test run from eclipse that I have been using to post off to my test server. This produces the error;

Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:174)

at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:238)

at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:289)

4.2) The reason for this is that eclipse is not referring to the keystore that we have created. At the time of writing I have not sussed out how to make eclipse use this - Do you know ?

So instead I have added the same two certs to javas keystore

4.3) cd to the directory where java's keystore is held. This might be something like:

C:\java\jdk1.6.0_18\jre\lib\security

4.4) the keystore is a file called cacerts

4.5) copy the files c:\ssl\server.crt and c:\ssl\tomcat.cer to this directory

4.6) import the server CA into the java keystore

keytool -import -alias serverCA -file server.crt -keystore cacerts

4.7 import tomcats cert into the java keystore

keytool -import -alias tomcat -file tomcat.cer -keystore cacerts

5) Test your app again.

Hopefully all will be hunkdory. Enjoy.

6) Caveats:

6.1) the passwords for all keystores and certs are 'changeit'. this is the default keystore password

and I suggest you change this for a production system

6.2) Using a self signed cert is great for a test environment of for a private system but not for a commercial released application. For this you will need to get & pay for a signed certificate from an approved authority such as Verisign.

train times using RESTful urls

While travelling by train in this arctic winter I came across this really nice simple train times site using RESTful URLs to great effect.

http://traintimes.org.uk/blackburn/entwsitle/16:00/today

The output is really clear and simple making it great for display on small phone screens.

Apache Ivy - Working with the Apache tutorial

Apache Ivy is a dependancy manager and appears to be flavour of the month in some circles and claims to be less complicated than Maven.

Here is a quick tutorial that expands on Apaches tutorial page. They provide one very small build.xml in a style that they say is not best practise for a number of reasons:

1. it is standalone
2. it contains the code within the build.xml
3. it downloads ivy locally instead of using a remote installation

I struggled to find a simple example of a build in the accepted syntax so I will attempt here to describe how this build.xml can be changed in 4 steps to ultimately work in the conventional style with a build.xml and ivy.xml files and a remote installation of ivy.

My next project will be to 'ivyfy' a spring project. Hopefully I will be successfull - watch this space.

prerequisites:

Apache Ant installed to 1.6 or greater

1) The basic Tutorial from Apache Ivy

• Essentially follow their instructions & download the build.xml from the tutorial page
• I had to edit the downloaded file to remove the dashes at the start of some lines
• run 'ant'
• I also changed the following line to refer to the latest formal release of ivy, but both will work.

from:

 <property name="ivy.install.version" value="2.0.0-beta1" /> 

to:

    <property name="ivy.install.version" value="2.1.0" />

2) Tutorial with no ivy

• One of things about ivy is that you are supposed to take an existing project and 'ivyfy' - IE remove your local third party jars and rely on ivy to fullfill your dependancies
• So this tutorial simply takes the results from the previous example and gets it to work without ivy.
• It therefore needs the src folder & its contents, the commons-lang jar in a lib dir in the project and a build.xml
• The file structure for this and future tutorials is:

.
./build.xml
./file
./filestructure
./lib
./lib/commons-lang-2.1.jar
./src
./src/example
./src/example/Hello.java

• The build.xml required is:

<project name="go-ivy" default="go" >

<property name="build.dir" value="build" />
<property name="src.dir" value="src" />
<property name="lib.path.id" value="lib" />

<path id="compile.classpath">
<fileset dir="${lib.path.id}">
 <include name="**/*.jar"/>
</fileset>
</path>

<!-- =================================
     target: go Go ivy, go!
    ================================= -->
<target name="go" depends=""
       description="--> resolve dependencies, compile and run the project">
   <echo message="NOT using ivy to resolve commons-lang 2.1..."/>

   <echo message="compiling..."/>
   <mkdir dir="${build.dir}" />
   <javac srcdir="${src.dir}" destdir="${build.dir}" >
      <classpath refid="compile.classpath"/>
   </javac>

   <java classname="example.Hello">
       <classpath>
          <fileset dir="${lib.path.id}">
            <include name="**/*.jar"/>
          </fileset>
          <path location="${build.dir}" />
       </classpath>
   </java>
</target>


<!-- =================================
     target: clean         
    ================================= -->
<target name="clean" description="--> clean the project">
   <delete includeemptydirs="true" quiet="true">
       <fileset dir="${src.dir}" />
       <fileset dir="${build.dir}" />
   </delete>
</target>
</project>

3) Tutorial with ivy.xml

• In the above example everything is defined in the build.xml. The clever thing with ivy and maven is that your project does not explicitly contain third party jars.
• Instead these are defined in a config file, ivy.xml in this case.
• This step moves the dependancies defined in build.xml into a seperate ivy.xml
• again simply run 'ant' to test.
• For this and subsequent tutorials I did not generate the src files. I kept them from the first tutorial
• Therefore for this and subsequent tutorials all you need is the src folder with the src file, the build.xml & ivy.xml
• Here is the ivy.xml

<ivy-module version="2.0">
<info organisation="Utilisoft" module="Test-withivy.xml"/>
<dependencies>
<dependency org="commons-lang" name="commons-lang" rev="2.1"/>
</dependencies>
</ivy-module>

• Here is the new build.xml - I have removed much of the comments to make it smaller.

<project name="go-ivy-with-ivy.xml" default="go" xmlns:ivy="antlib:org.apache.ivy.ant" basedir=".">
<!-- here is the version of ivy we will use. change this property to try a newer
version if you want -->
<property name="ivy.install.version" value="2.1.0" />
<property name="ivy.jar.dir" value="${basedir}/ivy" />
<property name="ivy.jar.file" value="${ivy.jar.dir}/ivy.jar" />

<property name = "ivy.lib.dir" value="${basedir}\newlib" />

<path id="lib.path.id">
  <fileset dir="${ivy.lib.dir}" />
</path>

<property name="build.dir" value="build" />
<property name="src.dir" value="src" />

<!-- =================================
target: go Go ivy, go!
================================= -->
<target name="go" depends="resolve "
description="--> resolve dependencies, compile and run the project">
<echo message="using ivy to resolve commons-lang 2.1 with ivy.xml"/>

<echo message="compiling..."/>
<mkdir dir="${build.dir}" />
<javac srcdir="${src.dir}" destdir="${build.dir}" classpathref="lib.path.id" />

<java classname="example.Hello">
<classpath>
  <path refid="lib.path.id" />
  <path location="${build.dir}" />
</classpath>
</java>
</target>


<!-- =================================
target: clean
================================= -->
<target name="clean" description="--> clean the project">
<delete includeemptydirs="true" quiet="true">
<fileset dir="${src.dir}" />
<fileset dir="${build.dir}" />
</delete>
</target>

<target name="resolve" depends="install-ivy" description="Resolve the dependencies">
<ivy:retrieve/>
</target>


<!-- =================================
target: install-ivy
this target is not necessary if you put ivy.jar in your ant lib directory
if you already have ivy in your ant lib, you can simply remove this
target and the dependency the 'go' target has on it
================================= -->
<target name="install-ivy" depends="download-ivy" description="--> install ivy">
<!-- try to load ivy here from local ivy dir, in case the user has not already dropped
it into ant's lib dir (note that the latter copy will always take precedence).
We will not fail as long as local lib dir exists (it may be empty) and
ivy is in at least one of ant's lib dir or the local lib dir. -->
<path id="ivy.lib.path">
<fileset dir="${ivy.jar.dir}" includes="*.jar"/>
</path>
<taskdef resource="org/apache/ivy/ant/antlib.xml"
    uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>
</target>

<target name="download-ivy" unless="skip.download">
<mkdir dir="${ivy.jar.dir}"/>
<!-- download Ivy from web site so that it can be used even without any special installation -->
<echo message="installing ivy..."/>
<get src="http://repo1.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar" dest="${ivy.jar.file}" usetimestamp="true"/>
</target>
</project>

4) Tutorial with remote Ivy

• Hopefully you are still with me. This is the last step in this tutorial set.
• The above example still downloads ivy locally
• This example will now use ivy as part of your apache and installation
• In the last example a ivy.jar file appeared in an 'ivy' directory.
• Determine the location of your apache ant installation and copy this jar into the apache ant lib directory.
• Ensure you have an ANT_HOME environment variable pointing to the apache4 ant installation directory
• Below is the new build.xml file you will need.

<project name="go-ivy-with-ivy.xml" default="go" xmlns:ivy="antlib:org.apache.ivy.ant" basedir=".">
<!-- here is the version of ivy we will use. change this property to try a newer
version if you want -->
<property name="ivy.install.version" value="2.1.0" />
<property name="ivy.jar.dir" value="${basedir}/ivy" />
<property name="ivy.jar.file" value="${ivy.jar.dir}/ivy.jar" />

<property name = "ivy.lib.dir" value="${basedir}\newlib" />

<path id="lib.path.id">
       <fileset dir="${ivy.lib.dir}" />
</path>

<property name="build.dir" value="build" />
<property name="src.dir" value="src" />

<!-- =================================
 target: go
           Go ivy, go!
================================= -->
<target name="go" depends="resolve "
   description="--> resolve dependencies, compile and run the project">
<echo message="using ivy to resolve commons-lang 2.1 with ivy.xml"/>

<echo message="compiling..."/>
<mkdir dir="${build.dir}" />
<javac srcdir="${src.dir}" destdir="${build.dir}" classpathref="lib.path.id" />

<echo>
We are now ready to execute our simple program with its dependency on commons-lang.
Let's go!
</echo>
<java classname="example.Hello">
   <classpath>
       <path refid="lib.path.id" />
       <path location="${build.dir}" />
   </classpath>
</java>
</target>

<!-- =================================
 target: clean     
================================= -->
<target name="clean" description="--> clean the project">
<delete includeemptydirs="true" quiet="true">
   <fileset dir="${src.dir}" />
   <fileset dir="${build.dir}" />
</delete>
</target>

<target name="resolve" depends="" description="Resolve the dependencies">
<ivy:retrieve/>
</target>
</project>

And to the future. I have two immediate aims:

1. Get a build of a spring hibernate using Ivy
2. Get a project with dependant projects using spring and hibernate using ivy.

oracle - listing all tables in a schema

select table_name,

to_number( extractvalue( xmltype(

dbms_xmlgen.getxml('select count(*) c from '||table_name))

,'/ROWSET/ROW/C')) count

from user_tables;

courtesy of Burleson

SQL info for oracle/sqlserver/mysql

While performing some testing to ensure my app was compliant with multiple db flavours I came across this site.

It is a useful reference to quickly see how the different flavours perform some common operations.

http://www.sqlinfo.net/

humax PVRs and the digital switch over

I have two Humax PVR freeview recorders that have been running quite happily for several years.

The 4th November was our official DSO date for the north west and suddenly most of my BBC channels had gone.

My two PVRs are a new 9300T and an aging 8000T.

A retune on the 9300 brought BBC back but with BBC1 on channel 800. The 8000T could not find BBC anywhere.

The fix for the 9300 was to go to the Installation Menu, select the 'Default Setting' menu, enter the pin code and reset the box. Low and behold BBC1 was back on channel 1.

However the 8000 was proving more awkward. For this box I performed a software update request from tyhe system menu. Despite not saying that any of the versions had changed this fixed the problem and BBC1 is now back on channel 1.

In parrallel I did email humax on UKSupport@humax-digital.co.uk and congratulations to them I did get a very prompt reply but I had fixed the problem by the time I had read the email. For interest they said to do the following:

Please follow the steps below to perform a Default reset;

1. Power OFF the receiver

2. Disconnect the Aerial cable

3. Power On the receiver

4. Press MENU

5. Select Installation

6. Enter your password (default = 0000)

7. Select Default Setting

8. Select YES

9. Enter your password (Default = 0000)

10. When the receiver restarts, power off

11. Connect the Aerial Cable

12. Power ON - the receiver will then search for the channels.

Eclipse, workspaces and JREs

Be aware that installed JREs are workspace specific.

IE. Your default JRE might be the MyEclipse 6.0.

You install and add Java 1.5 JRE and make this your default.

Then three months later you create a new workspace.

Low and behold this new workspace does not know about the 1.5 JRE.