vrijdag 18 december 2015

A bit of Reflection

Introduction

In all the years I have been programming one of the dullest things I found to do was writing and maintaining converters. I had a couple of discussion about the subject. The point that always came up as a defence to not use bean mappers like Dozer was the fact that they used Reflection under the hood. I agree up to some level that the use of Reflection in production code should not be used in production code. The interesting point is that most of the third party libraries are using Reflection under the hood. Every library that uses Annotations for example are using it. Think about Spring, all the JPA libraries, log4j, Junit etc...
So why not for something like bean mapping. So lets see if we can build one our selfs.


The solution

A bean mapper is meant to  copy values from one bean to the other. So what you need to accomplish this is to know the field names of the "other" bean.

 The solution is actually not that complicated What I did was create a couple of Annotations and a handler.

If the field names are different you can pass the name of the fields in the "other" bean by using an annotation.
The Annotations look like this:


//The rentention policy tells when this Annotation should be used.
@Retention(RetentionPolicy.RUNTIME)
//The target tells where the annation should be placed. In this case field only.
@Target(ElementType.FIELD)
public @interface Copy {
    // the string that contains the value given to the Annotation.
    public String copyTo();

}

If the field names are the same you can do it without an annotation. 
I show that later.

If you dont want the fields to be copied you can use simply another annotation that prevents the field to be part of the copy process:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface IgnoreCopy {
}

The annotation in use look something like this:

// copying with a different field name
@Copy(copyTo = "testString2")
private String testString;
// copying wit the same fieldname
private String testStringNoAnnotation;
 
should not be copied.
@IgnoreCopyprivate String notToCopy;


To copy the values first of all you need the fields of the bean to copy from:
//Use the getDeclaredFields instead of the getFields. Otherwise you can't find 
//the field you are searching for. 
Field[] fields = copyFrom.getClass().getDeclaredFields();

//looping through the fields to find the annotation if needed.
for(Field field : fields) {
    //The annotation if the field names are different.
    Copy copy = field.getAnnotation(Copy.class);
    //The annotation to see if the field shouldn't be copied. 
    IgnoreCopy ignoreCopy = field.getAnnotation(IgnoreCopy.class);
    if(ignoreCopy == null) {
        if ( copy != null) {
            //the actual method to copy with different field name 
            addValueToField(copyFrom,copyTo, field, copy.copyTo());
        } else {
            //the actual method to copy with the same  field name 
            addValueToField(copyFrom, copyTo, field, field.getName());
        }
    }
}


private void addValueToField(final Object copyFrom, 
final Object copyTo, final Field field, final String name) {
    try {
        // Searching for the field in the "other" bean
        Field copyToField = copyTo.getClass().getDeclaredField(name);
        // The fields are private so we make them accessible
        copyToField.setAccessible(true);
        field.setAccessible(true);
        // The actual copying of the values
        copyToField.set(copyTo, field.get(copyFrom));
        //stop the accessibility of the fields. 
        copyToField.setAccessible(false);
        field.setAccessible(false);
    } catch (NoSuchFieldException e) {
        log.error("could not find field " + name);
        e.printStackTrace();
    } catch (IllegalAccessException e) {
        log.error("could not access field " + name);
        e.printStackTrace();
    }
}
 
 
 
// I added two more methods that are helpfull with the cherry picking. 
  
/** * checks if ignorefield is set in the target.class 
* @param copyTo the target class. 
* @param fieldName the fieldname to be set. 
* @return a boolean if the field is right to set. 
*/
 private boolean ignoreCopyToField(final Object copyTo, final String fieldName) {
    Field field = null;
    // A list of fieldnames to prevent the NoSuchFieldException
         List<String> fieldNames = checkFieldExists(copyTo);
        IgnoreCopy ignoreCopy = null;
        try {
            // check if the field exists in the target class. 
           if (fieldNames.contains(fieldName)) {
                // retrieve the field with the name 
                field = copyTo.getClass().getDeclaredField(fieldName);
                // retrieve the annotation from the field
                ignoreCopy = field.getAnnotation(IgnoreCopy.class);
            }
        } catch (NoSuchFieldException e) {
            e.printStackTrace();
        }        return  ignoreCopy != null;
}

/** * creates a list that has all the fieldnames in it. 
* @param copyTo the target class 
* @return a list that has all the fieldnames in it 
*/ 
private List<String> checkFieldExists(Object copyTo) {
    List<String> fieldNames = new ArrayList<String>();
    Field[] fields = copyTo.getClass().getDeclaredFields();
    for (Field field1: fields) {
        fieldNames.add(field1.getName());
    }
    return fieldNames;
}
 

Conclusion

The choice to be made is more of a philosophic one. Is it worth to add some more reflection to the project at hand. I think the decision is to use it where it is useful. If it can save you writing and maintaining miles of boring code my advice would be to do it. In many cases it allready happens. So think about and work this out for your self. The code to build this is short and simple.

The original source code can be downloaded  here
The jar file can ben downloaded  here
Have fun!


dinsdag 10 februari 2015

Sorting data

Introduction:

In the issue I had at hand we needed to sort the data to create a matrix. So actually we needed to morph a list to a matrix form. 

The problems:

At first hand I was thinking about using the states of the data to figure out where in the matrix it should find its place. As soon I made a drawing about my idea, I stopped working at it. I needed to create a complex structure with a lot of if statements. I understood that, if I choose this way I might run into a lot of bugs. I needed to think harder. 

The solution:

I turned the whole idea around and looked at it from the other side. Instead of creating the matrix directly from the list I could something like a pre-sort. The next question obvisiouly would be: How the hell do I do that? The answer was actually quite simple looking back at the issue. I took a LinkedhashMap and a Treemap. I made sure that the query returned everything sorted by date. 

In the LinkedHashmap I used as a key the date and the Treemap. The LinkedHashmap keeps things in order of incoming. So it kept the order of the query I created by date. The second ordering would be by currency  where the payment was in done.  This I used as a key in the TreeMap together with the rest of the object. The Treemap sorts on aliphatic or numeric order using the black and red sorting algorithm. 

Now All I had to do was write a double loop for both of the maps and put the data I needed in place. 


Conclusion:

Sometimes it is better to make that drawing and have a better look at what you need to-do. I believe that part is the greatest joy in my job. Create something that gives me the proud feeling.

Have fun!


vrijdag 30 januari 2015

saving passwords the save way

Introduction:

Every time I search on security matters, I always find the things to prevent attacks from the outside. These issues are important there is no doubt about that. But the one issue that kept running around my mind was one of the jobs where they got into your database. That is the place where an unwanted visitor should stay away. One of the last resort solutions to prevent this unwanted visitor to steal your customer data is to hash the passwords and encrypt the data in that database. The downside of this technique is that most of the encrypting and hashing methods all ready have white papers. So the only thing is: that buys you a bit more time before they got to your data.

Part of the solution:

I did not solve the whole case of steeling data yet. I did found away to store passwords strongly protected in my database. The solution is quite simple: don't store the real password in the database but an abstraction of that password. Every character has an ASCII number changing the characters in the password into numbers gives you the possibility to calculate with them like this:


double key = 0;
char[] passwordChars= password.toCharArray();
for (char keyPart : passwordChars) {
      key += Character.getNumericValue(keyPart);
}

Now you have a nummeric value of a password. How far you will calculate this is upto you.
The next question is ofcourse what to do with a number. The simple thing is that you can allready store this number into the database and you have an abstraction of the password. But the fun is to push it a bit further. You can create an hashmap that has a double as a key and a String as a value.
With the calculated number you get the string value out of the database. and store that as a password.

Something like this:
private Map<Double, String> falsePasswords = new HashMap<Double, String>();
and fill it with nice long strings with a lot of weird characters:
falsePasswords.put(1.0, "*(_)((UIiuyuUITYFTYR%R&$%&%tituyutyr867987yuyuiyo8&)*0");
I would make this a lon list of false passwords and create a calculation on the password that gives a good range of false passwords.

To make it look like a real password you can hash it and make the unwanted visitor think that he can decrypt all your passwords.

Conclusion:

The solution to most of the problems are not that hard. In this case a simple calculation and a hashmap brought the solution.

Have fun!

maandag 12 januari 2015

Spring-boot rest/jpa full blown

Introduction

For a while I was looking for a suiting solution where I could use spring rest services combined with the Spring/JPA repositories. What I actual was looking for was a query driven rest service where I did not had to convert one bean to another. This awfull boilerplate code and is a maintainance nightmare.

After a lot of searching I ran into Spring-boot where the got a couple of steps further than I was looking for. After studying it for a while I got used to their ideas and must say I like them.

While I was searching I ran into a lot of examples but none of them was showing me how to deal with spring-boot in a professional and serious mather. So i picked up a lot from the reference guide and put the pieces of the puzzle together.

The configuration
I build the whole story in Maven for the gradle fans have a look at the reference guide. There is fully explained howto do it in gradle.

The basiscs of spring-boot is the simple fact that they provide you with a super pom for maven where you can cherry pick the parts you really like to use. Here are the cherries I picked from that tree:

The project pom 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <parent>
        <artifactId>abstractservice</artifactId>
        <groupId>com.purat.services</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <artifactId>userservice</artifactId>
    <packaging>jar</packaging>

    <properties>
        <tomcat.version>8.0.3</tcat.version>
        <!-- use UTF-8 for everything -->
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <start-class>com.purat.Application</start-class>
        <spring-boot-version>1.1.10.RELEASE</spring-boot-version>
    </properties>
// The super pom provided by spring-boot
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.1.10.RELEASE</version>
    </parent>

    <dependencies>
// these are the depedency to deliver all the necessary jpa libraries to approach the database. 
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency
//This is the library to deliver all the necassary rest libraries. To create for example your own endpoints
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-rest</artifactId>
        </dependency>
//This is the library that gives us the web application part. 
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
//This is the library that gives us the production-ready parts. For more details look at boot-starter-actuator
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
//For the database I used postgress.
        <dependency>
            <groupId>postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <version>9.0-801.jdbc4</version>
        </dependency>
// For handling the getters and setters I used lombock.
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
                <version>1.12.6</version>
                <scope>compile</scope>
        </dependency>
    </dependencies>   <build>
        <pluginManagement>
        <plugins>
//This plugin is responsible for the kind of packaging you desire. You can choose  a war or a jar
This plugin will build it with the desired content.  
To deploy a war file there are still a couple of steps to take. You can find the description for that here:
traditional deployment
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
        </pluginManagement>
    </build>

    <repositories>
        <repository>
            <id>spring-releases</id>
            <name>Spring Releases</name>
            <url>https://repo.spring.io/libs-release</url>
        </repository>
        <repository>
            <id>org.jboss.repository.releases</id>
            <name>JBoss Maven Release Repository</name>
            <url>https://repository.jboss.org/nexus/content/repositories/releases</url>
        </repository>
    </repositories>

    <pluginRepositories>
        <pluginRepository>
            <id>spring-releases</id>
            <name>Spring Releases</name>
            <url>https://repo.spring.io/libs-release</url>
        </pluginRepository>
    </pluginRepositories>
</project>

The Application class and application.properties

Despite of what you see on the internet I like to keep the Application class to a bare minimum. We need this baby only to start the project up after we build a war or a jar. 


import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;

/**
 * Created by compurat on 1/11/15.
 */
//Indicates that a class declares one or more @Bean methods and may be processed by the Spring container to generate bean definitions and service requests for those beans at runtime.
@Configuration
// This annotation configures  and uses the standard beans that are on the classpath of the diverse libraries.
@EnableAutoConfiguration
//Configures component scanning directives for use with @Configuration classes.
@ComponentScan
public class Application {

    public static void main(String[] args) {
//This is the only line you need to start up you application.
        ConfigurableApplicationContext context = SpringApplication.run(Application.class);
    }

}

The application.properties file under the resources folder:

The way this works, looks to me like the Wicket framework. 
//The configuration of the database
spring.datasource.url=jdbc:postgresql://localhost:5432/people
spring.datasource.username=Pieter
spring.datasource.password=Pieter01
spring.datasource.driver-class-name=org.postgresql.Driver
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
hibernate.hbm2ddl.auto=create-drop
spring.jpa.show-sql=true
// Setting the end point properties give it an id,give the possibility to shut the endpoints down and sensitive is security wise. Does an endpoint need username and password to be approached.
endpoints.beans.id=springbeans
endpoints.beans.sensitive=false
endpoints.shutdown.enabled=true
//Tomcat serverlog enabling and patterns.
server.tomcat.access-log-enabled=true
server.tomcat.access-log-pattern=%a asdasd
The log file propeties.
logging.file= logging/userservice.log
logging.level.org.springframework.web: DEBUG
logging.level.org.hibernate: ERROR

Basicly we now have ready to run spring-boot webservice with JPA. It does not work yet we still need to fill in the missing pieces here. 

The application
Lets start with the rest endpoint. That is where it all begins. This is the profile endpoint and does not take any parameters in this case. But it give you an idea how an endpoint in spring-boot works:

import com.purat.data.People;
import com.purat.data.repository.PeopleRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

/**
 * Created by compurat on 1/11/15.
 */
//Tells spring that this is obvious a rest controller
@RestController
//Tells spring what the base url is.
@RequestMapping("peopleservice")
public class ProfileEndpoint {
    @Autowired
//spring/jpa repository to approsch the database
    private PeopleRepository peopleRepository;
//Tells spring what the endpoint url is.
    @RequestMapping("/profile")
    public Person profile() {
   The jpa entity
    Person person 
//The query parameter names are in the repository method name. in this case EmailadressAndPassword
= peopleRepository.findByEmailadressAndPassword("pieter.roskam@gmail.com","Roskam01");
If you Return the entity in the rest controller spring will underwater serialize the object to an json file.
        return person;
    }

As second part I like to handle the jpa entity:
import lombok.Getter;
import lombok.Setter;

import javax.persistence.Entity;
import javax.persistence.Id;

/**
 * Created by compurat on 1/11/15.
 */
//Lombock getters and setters keep you code nice and clean.
@Getter
@Setter
//What table to approach
@Entity(name="people")
public class Person {
// the unique id and all the fields as named in the table.
    @Id
    private long id;
    private String firstname;
    private String lastname;
    private String address;
    private String housenumber;
    private String postalcode;
    private String city;
    private String telephone;
    private String verification;
    private String emailadress;
    private String password;

}

As third and last part I would like to show the jpa repository:
package com.purat.data.repository;

import com.purat.data.User;
import org.springframework.data.repository.CrudRepository;

 
The spring/jpa repository gives some benefits. For simple queries you can use the parameter names in the method, but for more complicated queries you can also use the @Query annotation to write a jql query .
public interface PeopleRepository extends CrudRepository<Person,Long> {

    User findByEmailadressAndPassword(String emailadress, String password);
}

Conclusion

Spring-boot took a complete new road to developing and deploying rest services. The most interesting part is probably that you can have a rest service in a jar file. That was a mind boggle to me for a while. The secret lies in the fact that it is a self executing jar. While starting up your Application class it will also startup tomcat if you configured it right (dependencies and properties file).

The second part that might be of interest, is the fact that it looks like that the separation of tiers is gone. But if you look closely it is not:

1. The separation of the database is still intact because it is hidden behind the spring/jpa repositories. They are not reachable from the outside. 

2. The entity bean seams to be exposed at the frontend but again that is not completely true. The bean will be serialized to a json file by Spring. That is the only visible thing on the outside. The only one that should reach this service is the frontend. Which is also separate.

I love the part that you can package a rest service into a jar. You put it somewhere on your server and start it with java -jar servicewhatever.jar. That will fire up everything you build for this rest service.


have fun!