json

Golang Reflection and Interfaces

Before diving into interfaces and reflection I want to give a bit of a background on the use case I had to apply them to.

Manage Struct metadata as form of JSON components

In most data designs data is still relational. Non Relational databases have their use case but when dealing with Account-User Management, RBAC and other relational data models a relational database still is the best tool of choice.
In an iterative development process all columns of a table might not be known before hand and having a framework to quickly iterate on top becomes very handy. For example; instead of having to add new columns to a table the concept of a JSON component column can be used where data points that are not searched can be stored in a JSON string, which allows for the data model to be defined at the application level. Projects like OpenStack and Rancher already follow that strategy.
UPDATE: MySQL version 5.7.8 introduced native support for JSON: https://dev.mysql.com/doc/refman/5.7/en/json.html

Implementing JSON components in Go

A StructTag can be used to define which attributes should be stored in the JSON component.
When persisting the struct to the database the component attributes would be added to the JSON Components string and only the Components string would be persisted to the database.

type Stack struct {  
    StringComponent string  `json:"stringComponent,omitempty" genesis:"component"`
    IntComponent    int `json:"intComponent,omitempty" genesis:"component"`
    BoolComponent   bool    `json:"boolComponent,omitempty" genesis:"component"`
    FloatComponent  float64 `json:"floatComponent,omitempty" genesis:"component"`
    Components  string
}

First attempt at Reflection

At first I created a method in the Stack type which would iterate over all its attributes and build up a JSON string based on the attributes that were tagged with the component StructTag

func (s *Stack) prep() {  
    components := map[string]interface{}{}
    fields := reflect.TypeOf(s).Elem()
    values := reflect.ValueOf(s).Elem()
    for i := 0; i < fields.NumField(); i++ {
        field := fields.Field(i)
        value := values.Field(i)
        if isComponent(field) {
            components[field.Name] = value.Interface()
        }
    }
    c, err := json.Marshal(components)
    if err != nil {
        fmt.Printf("Error creating components JSON object %v\n", err.Error())
    }
    jsonString := string(c[:])
    s.Components = jsonString
}

Go Playground sample

The main problem with the approach listed above is that the prep method is tied to the Stack struct and other structs can't reuse it.

Second attempt at Reflection

Instead of calling prep as a struct method, by making Prep a public function then any struct can be provided as an argument and the method will take care of building the JSON component string via reflection.

func Prep(obj interface{}) {  
    components := map[string]interface{}{}
    fields := reflect.TypeOf(obj).Elem()
    values := reflect.ValueOf(obj).Elem()
    for i := 0; i < fields.NumField(); i++ {
        field := fields.Field(i)
        value := values.Field(i)
        if isComponent(field) {
            components[field.Name] = value.Interface()
        }
    }
    c, err := json.Marshal(components)
    if err != nil {
        fmt.Printf("Error creating components JSON object %v\n", err.Error())
    }
    jsonString := string(c[:])
     values.FieldByName("Components").SetString(jsonString)
}

Go Playground sample

Some important points to keep in mind:

  • Go has static and underlying types
  • Difference between Types and Kinds:
    • A Kind represents the specific kind of type that a Type represents.
    • In other words, a Kind is the underlying type and the Type is the static type
    • Example: http://play.golang.org/p/jAczBqbx0a

Even though the obj argument is an empty interface, its Kind at runtime will be of Pointer and its Type of *Stack (or whatever struct is passed in)
That is important to understand since to manipulate struct fields with reflection you need to have a Struct Kind.
For example; the following statement would panic:

reflect.ValueOf(obj).NumField()  

or

reflect.ValueOf(obj).Field(0)  

Accessing a Struct from a Pointer

Initially this might seem a bit alien, however, I like to relate to how you can deference pointers in C.
In C you can use the -> notation to access the values a pointer points to:

#include<stdio.h>

struct Stack {  
  int x;
};

int main(void) {  
  struct Stack s; 
  struct Stack *p;
  p = &s;
  s.x = 1;
  printf("s.x %d\n", s.x); // 1
  printf("&s %p\n", &s); // address of s
  printf("p %p\n", p); // address of s
  printf("&p %p\n", &p); // address of pointer
  printf("*p.x %d\n", p->x); // 1
  return 0;
}

In Golang the concept is similar but the syntax is a bit different. You can use the Elem() method which returns the value the interface contains. In the case of a pointer it will return the object that is being referenced to.

reflect.ValueOf(obj).Elem()  

and since the pointer is pointing to a Struct the Field method can be utilized

reflect.ValueOf(obj).Elem().NumField()  
reflect.ValueOf(obj).Elem().Field(0)

Jackson overview

Recently I found the need to dig deeper and start getting a better grip on how jackson handles data parsing and manipulation. I always had some problems converting from JPA objects -> Date -> DateTime -> Json and back and forth. However, by plugin in some custom serializers/deserializers I always hacked my way around it.

Now that I’m starting to use Jongo I had to look at some features jackson provides to customize the mapping between mongo BasicDBObjects and POJOs

Jackson overview

One thing to make clear is that the official project is now being maintained under FasterXML, not Codehaus.
Codehaus is a collaborative environment for building opensource projects, jackson moved away from codehaus is now under the FasterXML umbrella

The last release made under the codehaus banner was in July 14 2013, version 1.9.13,
Some differences between the old and new jackson are:

  • Maven build instead of Ant
  • Annotations carved out to a separate package
  • Java package is now com.fasterxml.jackson.core instead of org.codehaus.jackson

Main components

jackson-core – defines low-level streaming api, and includes JSON-specific implementations
The core package is where all the low level parser implementation is kept, some core classes that handle the raw json object creation are JsonReadContext and JsonWriteContext

More info can be found at their javadocs

jackson-annoations – contains standard jackson annotations
The annoations packages contains the definition of all the annoations used by jackson, 31 in total.
Some annoations worth noting are:

  • @JsonProperty
  • @JsonIgnore
  • @JsonInclude
  • @JsonView
  • @JacksonInject
  • @JsonSubTypes

jackson-databind – implements data-binding and object serialization support on streaming packages

This is the package the handles most of the jackson parsing logic, classes like the ObjectMapper and SimpleModule are kept in this packaged

The databind package bootstraps the annotations defined in the jackson-annotations package, one reason to separate the annoations from the databind package is to allow third party libraries to extend the utilize the annoations without having to include the whole databind package in its build.

Besides the main modules jackson also provides support for third party libraries, some of them are:

  • joda
  • hibernate
  • guava
  • hppc
  • jax-rs

Overall jackson is a stable library that provides solid data manipulation support for different data types in java. The way it is architectured it allows new types to be easily implemented and it also provides a rich feature api that developers can extend to fit different application needs.

One thing I would say is that the documentation about jackson is very fragmented, I usually like to go to a single page and get all the info I need about a project, with jackson I always find myself hoping around between sites to find the info I need. It might be fragmented due to the change from codehaus to fasterxml, but in any case I would really like to see some effort into making the library more presentable.

I know for sure that several applications use jackson and the library is rock solid, its web presence should reflect the same.

A good example would be : Jongo
Jongo documentation is pretty well done, plus other useful links regarding the project are well organized. Jongo uses jackson as its base parser, maybe jackson could use some of jongo docs ideas and improve its own.