Dynamically downloading a JAR and adding to classpath

I recently had a situation where I had a JAR that had dependencies on another JAR, but wouldn’t always be packaged with the other JAR.  I’m not completely happy with what I ended up doing, but since I succeeded in doing what I wasn’t certain was possible, I decided to document it.  Basically (as the title suggests), I tweaked my code so that at runtime it will download the JAR and add it to the classpath.

Downloading from a URL

Normally I’d use something like the Apache Commons IO library to help with downloading the JAR, but since that’s one of the pieces in the JAR to be downloaded, I’m in a catch-22 situation.  Instead, I used vanilla Java for the implementation.  I ran into some minor complications because the server hosting the JAR did not have a signed certificate, so I had to force Java to ignore certificate errors.  Fortunately, no authentication was required–otherwise things would have been a bit more complicated.  Here is my class:


package com.nathanbak.gomi;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

public class UrlDownloader {

    /**
     * Downloads the specified URL to the specified file location. Maximum size
     * allowed is <code>Long.MAX_VALUE</code> bytes.
     *
     * @param url
     *            location to read
     * @param file
     *            location to write
     * @throws NoSuchAlgorithmException
     * @throws KeyManagementException
     * @throws IOException
     */

    public void download(URL url, File file) throws NoSuchAlgorithmException, KeyManagementException, IOException {
        TrustManager [] trustManagers = new TrustManager [] { new NvbTrustManager() };
        final SSLContext context = SSLContext.getInstance("SSL");
        context.init(null, trustManagers, null);
       
        // Set connections to use lenient TrustManager and HostnameVerifier
        HttpsURLConnection.setDefaultSSLSocketFactory(context.getSocketFactory());
        HttpsURLConnection.setDefaultHostnameVerifier(new NvbHostnameVerifier());
       
        InputStream is = url.openStream();
        ReadableByteChannel rbc = Channels.newChannel(is);
        FileOutputStream fos = null;
       
        try {
            fos = new FileOutputStream(file);
            fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
        } finally {
            if (fos != null) {
                fos.close();
            }
            is.close();
        }
    }

    /**
     * Simple <code>TrustManager</code> that allows unsigned certificates.
     */

    private static final class NvbTrustManager implements TrustManager, X509TrustManager {
        @Override
        public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException { }

        @Override
        public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException { }

        @Override
        public X509Certificate[] getAcceptedIssuers() {
            return null;
        }
    }
   
    /**
     * Simple <code>HostnameVerifier</code> that allows any hostname and session.
     */

    private static final class NvbHostnameVerifier implements HostnameVerifier {
        @Override
        public boolean verify(String hostname, SSLSession session) {
            return true;
        }
    }
}

Adding JAR to classpath

This was my first foray into dynamically changing the classpath at runtime.  I found many examples of how to load a specific class (when you know the full class name) from a JAR file, but there wasn’t as much information about stuffing a JAR of whatever into the current, running classpath.  After much trial and error, this is what I finally produced:


package com.nathanbak.gomi;

import java.io.File;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.MalformedURLException;
import java.net.URL;

public class JarAdder {
    public void addJarToClasspath(File jar) throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException, MalformedURLException {
        // Get the ClassLoader class
        ClassLoader cl = ClassLoader.getSystemClassLoader();
        Class<?> clazz = cl.getClass();
       
        // Get the protected addURL method from the parent URLClassLoader class
        Method method = clazz.getSuperclass().getDeclaredMethod("addURL", new Class[] {URL.class});
       
        // Run projected addURL method to add JAR to classpath
        method.setAccessible(true);
        method.invoke(cl, new Object[] {jar.toURI().toURL()});
    }
}

The addJarToClasspath() method wasn’t necessary on my Windows system.  My main JAR had a classpath to the dependency JAR specified in the MANIFEST.MF and as long as the JAR was downloaded there, it would be found.  However, on Linux it didn’t work and so the method is necessary (and it doesn’t seem to hurt anything on Windows).

Other thoughts

  • It is important to do the download and classpath changes before calling any code that depends on the stuff in the JAR.  Even imports in the same class can cause problems.
  • The downloadUrl() method is pretty generic and could be reused in a lot of situations provided the content being downloaded doesn’t get to big.
  • Different versions of Java seem to behave differently–I’ve only tested two Java’s so far (one on Windows and one on Linux), but have seen very different behaviour.
  • Since the certificate checking is disabled and code is loaded and at runtime, it seems like it would be an easy setup to attack or hack.
  • This method could potentially be used for applications to self-update without needing to restart.

Conclusion

I’m not sure this is a permanent solution for my problem, but it does work for the time being.  Also, I think the parts I learned while going through the process have potential to be used in future situations.

I love it when a plan comes together

After spending a lot of effort and encountering difficulties in creating pieces, I am often pleasantly surprised when the pieces come together quickly and easily.  This was the case for my latest home improvement tech project.  In my home, it seems like some areas are warmer than others–I realized that some variance will exist, but I wanted to reduce the overall difference between upstairs and downstairs.

The first step was to be able to measure the temperature or each area.  Thanks to my ESP8266 development boards, I am able to publish the upstairs temperature and publish it to a database and Bakboard.  With the new Nest thermostat and a little playing with the REST API, I was able to do something similar and publish the downstairs temperature to the BakBoard.  There are now four temperatures published on the Bakboard.

temps

I then wrote a simple Java program with that basically does the following:

  1. Get the temperature of the [Downstairs] thermostat
  2. Get the temperature of the [Upstairs] temperature sensor
  3. If the difference between the two temperatures is greater than 2 degrees, turn on the furnace fan

I had a little trouble figuring out how to turn on the fan, but this is the way I implemented it in Java:

public void runFan(String thermostatId, String authToken) throws Exception {
    final String rootUrl = "https://developer-api.nest.com";
    HttpPut httpPut = new HttpPut(String.format("%s/devices/thermostats/%s/fan_timer_active", rootUrl, thermostatId));

    StringEntity putEntity = new StringEntity("true");
    httpPut.setEntity(putEntity);
    httpPut.addHeader("Content-Type", "application/json");
    httpPut.addHeader("Authorization", "Bearer " + authToken);
        
    CloseableHttpClient httpclient = HttpClients.createDefault();
    try {
        CloseableHttpResponse response = httpclient.execute(httpPut);
            
        // We need to handle redirect
        if (response.getStatusLine().getStatusCode() == 307) {
            String newUrl = response.getHeaders("Location")[0].getValue();
            httpPut.setURI(new URI(newUrl));
            response = httpclient.execute(httpPut);
        }
           
        try {
            HttpEntity entity = response.getEntity();
            EntityUtils.consume(entity);
        } finally {
            response.close();
        }
    } finally {
        httpclient.close();
    }
}

Of course I want my code to run at regular intervals, but fortunately I had already figured out how to go about running a Java program every 15 minutes.  It was easy to toss everything into a Docker container and let it do its thing.

Here are a few notes/design decisions that I made when putting things together:

  • There are no changes to the basic functionality of the Nest thermostat.  It is not aware of the external temperature sensor and heats/cools as normal.  This means, even if something goes wrong in my code (or network connection or custom hardware or somewhere else), things can’t go too crazy.
  • My code does not control the length the fan runs–it starts the fan and lets the Nest take care of turning it off.  There is a default run time that can be set on the thermostat–in my case I set it to 15 minutes to match the run duration of my new program.
  • I have a two stage furnace and when just the fan is run it goes at half speed.  Even at full speed the furnace fan is pretty quiet, and at half speed we don’t even notice.
  • The thermostat only gives me the temperature in degree increments (if I were using Celsius it would be in half degree increments).  My homemade temperature sensor goes to greater precision, but it’s hard to say whether that greater precision provides better accuracy.  I went with a 2 degree variance threshold for enabling the fan to allow for rounding differences as well as accuracy differences between upstairs and downstairs temperatures.

As far as I can tell, everything came together smoothly and “just works” and has been for the past few weeks.  Occasionally I check the log to make sure it’s still running.  Once in awhile when I walk past the Nest I notice the fan icon indicating that the fan is running (and I can verify that by putting my hand near a vent).  The weather is still mild, so it will be interesting to see what happens when it gets colder (especially when I rev up the wood stove), but so far there seems less variance in temperature throughout the house.  I love it when a plan comes together . . .

Running a Java program every 15 minutes

I wrote a simple Java program that I wanted to run every 15 minutes.  I decided to wrap everything into a Docker image so that I would get the logging, restart capability, and portability goodness that one gets for free when running a Docker container.  It’s not a difficult thing to do, but it took me longer than it should since I made some incorrect assumptions.

Since I wanted the image to be small, I went with an “Alpine” version of the openjdk image.  My first incorrect assumption was that I could use cron and crontab like I do on Ubuntu or Red Hat systems–but Alpine doesn’t come with cron.  However, it’s actually easier than messing with crontab–I just had to put my script into the /etc/periodic/15min directory.

Once I had the script in place, I tried to run the container, but eventually discovered the the small Alpine image does not have the daemon enabled when the container starts up.  This was solved by running crond in the foreground.  Here’s a Dockerfile showing the important bits:

ROM openjdk:alpine
MAINTAINER Nathan Bak <dockerhub@yellowslicker.com>

# Create directory to store jars and copy jars there
RUN mkdir /jars
COPY jars/*.jar /jars/

# Copy bin directory of project to root directory
COPY bin/ /

# Copy runJavaApp script into correct location and modify permissions
COPY runJavaApp /etc/periodic/15min/
RUN chmod 755 /etc/periodic/15min/runJavaApp

# When the container starts, run crond in foreground and direct the output to stderr
CMD ["crond", "-f", "-d", "8"]

Here is the runJavaApp script:

#!/bin/sh
java -cp /:/jars/commons-codec-1.9.jar:/jars/commons-logging-1.2.jar:/jars/fluent-hc-4.5.2.jar:/jars/httpclient-4.5.2.jar:/jars/httpclient-cache-4.5.2.jar:/jars/httpclient-win-4.5.2.jar:/jars/httpcore-4.4.4.jar:/jars/httpmime-4.5.2.jar:/jars/jna-4.1.0.jar:/jars/jna-platform-4.1.0.jar com.yellowslicker.sample.Client

The “gotchas” I ran into with the script include:

  1. The script must begin with #!/bin/sh (out of habit I tried #!/bin/bash, but Alpine doesn’t come with bash)
  2. Each jar must be listed explicitly (I tried using /:/jars/*.jar for my path, but it didn’t work)

There are a lot of ways to schedule things, but this method was simple (once I figured it out) and I think it is robust.  In my case, it also fits well into the Docker microservice environment I’m running.