tirsdag den 16. august 2016

Using ssh without polluting your know hosts file

This command will allow you to ssh into a host without adding to the known hosts file:


ssh -o "UserKnownHostsFile /dev/null" -o StrictHostKeyChecking=no -i path/to/key ubuntu@12.34.567.890

mandag den 8. august 2016

How to solve Docker maven plugin failure to connect

The following error indicates that your default docker machine, does not exists, has been improperly configured, or is not connected to your shell.


[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.4.3:build (default-cli) on project foobar: Exception caught: java.util.concurrent.ExecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused 

The solution depends on which of the three reasons is the case. I usually try this:

eval "$(docker-machine env default)"

If this does not work I would delete the default machine and recreate it:
  1. Delete the default machine.
    docker-machine rm default
    
  2. Recreate the default machine.
    docker-machine create --driver virtualbox default
    
  3. Reconnect the shell to the default machine.
    eval "$(docker-machine env default)"
    

For more information see Docker.com get started.

mandag den 16. maj 2016

Serializing Custom Keys using Jackson

In this post I will show how to serialize a data transfer object (DTO) with a java.util.Map containing a POJO as key to JSON using the Jackson framework in a Jersey web application.

The DTO has a field of type Map with a key of type Foo like this:

import java.net.URI;
import java.util.HashMap;
import java.util.Map;

public class Dto {

    private Map<Foo, URI> fooUris;

    public Dto() {
        this.fooUris = new HashMap<>();
    }

    public ProvisioningDataDto(Map<Foo, URI> fooUris) {
        this.fooUris = fooUris;
    }

    public Map<Foo, URI> getFooUris() {
        return fooUris;
    }

    public void setFooUris(Map<Foo, URI> fooUris) {
        this.fooUris = fooUris;
    }
}

In this example we define Foo to be very simple:

public class Foo {

    private int id;

    public Foo() {
    }

    public Foo(int id) {
        this.id = id;
    }

    public int getId() {
        return this.id;
    }

    public void setId(int id) {
        this.id = id;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        Foo foo = (Foo) o;

        return getId() == foo.getId();

    }

    @Override
    public int hashCode() {
        return getId();
    }

}

We will use the Jersey client to send the DTO to the server. The serialization is done using the  the Dto using the javax.ws.rs.client.Entity class:

final Entity<DTO> json = Entity.json(dto);

where dto is DTO object.

Jackson will now complain on not knowing how to deserialize the map. Thus, we need to provide a custom key deserializer. Also the Foo key objects will be serialized by calling the toString method, which is rarely what we would like. Thus I will also show how to provide a key serializer.

First the deserializer. In this case it is very simple, and lacks some checks on the integrity of the data. We simply get the text of the JSON element (line 12) and convert it to an integer (line 13). I leave it as an exercise to the reader to implement proper checks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;

import java.io.IOException;

public class FooDeSerializer extends JsonDeserializer {

    @Override
    public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
        String str = p.getText().trim();
        return new Foo(Integer.valueOf(str));
    }
}

The serializer is equally simple in this case. We extract the id from the Foo object and use it as the field name of the JSON map element (line 11).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;

import java.io.IOException;

public class FooKeySerializer extends JsonSerializer<Foo> {
    @Override
    public void serialize(Foo value, JsonGenerator gen, SerializerProvider serializers) throws IOException, JsonProcessingException {
        gen.writeFieldName(value.getId() + "");
    }
}

The serializers must be enabled in order for Jackson to use them. This is done in two steps. First we add proper annotations to the Foo class. Then we register the serializers to the ObjectMapper used by Jersey.

First we add the annotations @JsonSerialize(keyUsing = FooKeySerializer.class) and
@JsonDeserialize(keyUsing = FooKeyDeSerializer.class) to the Foo class (line 1 and 2):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
@JsonSerialize(keyUsing = FooKeySerializer.class)
@JsonDeserialize(keyUsing = FooKeyDeSerializer.class)
public class Foo {

    private int id;

    public Foo() {
    }

    public Foo(int id) {
        this.id = id;
    }

    public int getId() {
        return this.id;
    }

    public void setId(int id) {
        this.id = id;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        Foo foo = (Foo) o;

        return getId() == foo.getId();

    }

    @Override
    public int hashCode() {
        return getId();
    }

}

The we must register the serializers using a Jackson module at the Jersey client:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
  ClientBuilder builder = ClientBuilder.newBuilder();
  builder = builder.register(JacksonFeature.class)
                   .register(new ContextResolver<ObjectMapper>() {

                               @Override
                               public ObjectMapper getContext(Class<?> type) {
                                   ObjectMapper objectMapper = new ObjectMapper();
                                   objectMapper.registerModule(new FooModule());
                                   return objectMapper;
                               }
                           });
        Client jerseyClient = builder.build();

The FooModule simply adds the serializers:

import com.fasterxml.jackson.databind.module.SimpleModule;

public class FooModule extends SimpleModule {

    public FooModule() {
        addKeySerializer(Foo.class, new FooKeySerializer());
        addKeyDeserializer(Foo.class, new FooKeyDeSerializer());
    }
}

Similarly we register the serializers at the server side:

  ResourceConfig packages = resourceConfig.packages("com.example.resources");
  packages.register(JacksonFeature.class)
          .register(new ContextResolver<ObjectMapper>() {

                      @Override
                      public ObjectMapper getContext(Class<?> type) {
                          ObjectMapper objectMapper = new ObjectMapper();
                          objectMapper.registerModule(new FooModule());
                          return objectMapper;
                      }
                  });

Now you will get nice looking JSON like this:

{"fooUris":{"1":"http://example1.com","2":"http:/example2.com","3":"http://example3.com"}}

torsdag den 21. januar 2016

søndag den 13. december 2015

JHead Rename images based on EXIF info

I just discovered JHead, a tool for manipulating JPEG images based on EXIT information. I don't know why I didn't find this tool before. This is extremely useful and convenient. I have tried a couple of tools on Windows and Mac but nothing is as convient at this one.

Install using homebrew:

brew install libjpeg
brew install jhead

libjpeg includes jpgtran, which is prerequisite for jhead but did not install as part of the jhead install.

I name my files using a date timestamp:

jhead -autorot -nf%Y-%m-%d-%Hh%Mm%S *.jpg

References:
http://www.phpied.com/installing-jpegtran-mac-unix-linux/
http://www.sentex.ca/~mwandel/jhead/
http://moinne.com/blog/ronald/bash/rename-your-photos-from-the-comandline-using-jhead

tirsdag den 8. december 2015

Release it

I am reading Release It by Nygard and these are some of the things to remember:


  • Use limits on queries. E.g. LIMIT 1 on Postgres or MySQL
  • Watch out for point-point connections
  • Set Socket.setSoTimeout() to protect against servers under heavy load
  • Use circuit breakers to degrade functionality gracefully under load
  • Return different errors for resource not available and application failures 
  • Fail fast, check availability of all required resources (ram, db, integration points, input)
  • Purge data (database, caches, logs)
  • Steady state
  • Bulkheads 
  • Cooperative demand control, handshaking between client and server to allow clients to back off. Problematic with http as the setup and teardown of connections is often the most expensive. Alternatively consider health checks either from client or from load balancer. Build handshaking into lowlevel protocols.