Thursday, August 23, 2012

Running applications using third party http services with out having to build stubs

While building applications which has integration with third party systems via http services the usual questions that arise are one, how to run your application in your own environment with out having to worry about installing the third party systems and how to test your system for different behavior that third party services expose. The first thing that comes to mind as a solution to this kind of a problem is to build some stubs. 

Is there a better way ? Can we avoid having to write stubs for every such kind of application which has lots of http services ?

Yes there is a simpler way, try fakerest. Fakerest is a sinatra based http stub server built using ruby. All you have to do is just write a configuration yaml with the http request URLs, response code, type, file that contains response body etc. A sample configuration file looks like: 

To use this all you have to do is install fakerest gem and start by running $ fakerest from command prompt. As you can see from the configuration file above, you can have as many services as you would like just by making a new entry in the configuration file with new request and response details.

The value for the content_file field in the configuration is the file from which the http response is loaded and served to the client. Fakerest supports all http methods like GET, POST, PUT, HEAD and more. With different configuration files you can just change the behavior of the http services just by restarting the Fakerest server by providing a different configuration file (this is quite handy for testing).

All this is fine, how do I verify what request is being posted to the http service ? 

Fakerest has an answer for this as well, all requests that are made to Fakerest are captured. You can just go to your browser hit http://host:port/requests/5 to see the last 5 requests, you can change the number to see the recent requests based on your need.

Not just simple POST and GET, with fakerest you can also upload files and verify how the file looks like on the server (fakerest) using the same requests url mentioned above. 

You could even change the http response status code to verify how your application behaves with different status codes that third party system might return. The same is true with the content type as well.

To know more details on how to configure and run go to fakerest github page.

Tuesday, August 7, 2012

Collaboration between agile and waterfall teams

This post shares my experience working on a project where multiple vendors are involved in developing a project which has lot of integration between the modules being built by different vendors. 

Interesting thing is the working methodology of each of these vendors. One vendor uses agile methodologies, while the other vendors software building process is completely waterfall model. 

This is how it all started, we had a inception with the clients to begin the project where we had a week of intense discussions on what the project is all about, what features do they need, technical components involved, prioritization, some high level flow diagrams and of course high level estimates (one thing I hate the most). We documented everything that came out of inception, got it reviewed and signed off by all parties involved and then got started with the development.

As this project has lot of integration with other modules which are being built by other vendor (at the same time) both the vendors went with the obvious approach of building stubs which simulates the modules the other vendor is building and finished development.

Sounds good, what is a problem here ? everything seems to be going so well as planned.

Well not really, to give a bit more context on what happened in inception around technical areas.

1. Teams identified all the components involved in building this
2. Came up with flow / sequence diagram which represents the user functional flow.
3. Agreed that integration between modules to be integrated will be over http.
4. Draft version (remember the word draft here) of the API documentation on all the modules that have to be integrated irrespective of which vendor is building it.
5. So on..

Both the vendors (Agile and Waterfall) took away these artifacts and started development. While for an Agile vendor all the artifacts from the inception were just a supporting material for development from initial discussions with clients and what is good enough to get the project started. But for the Waterfall vendor, these artifacts became a specification for the project.

Agile vendor as you all know would prefer design while you go, while developing stories API contracts were revisited by developers, changed based on whats the best choice for design etc. Started to communicate with the other vendor and they went crazy over it, the waterfall vendor were just so reluctant to agree to any changes to the API no matter if its for good design or not. 

For waterfall vendor, whatever was discussed and agreed in the inception (where just few hours were dedicated to discuss technical approach) became a specification for the project. Wonder how can a few hours of discussion is good enough for the vendor to freeze on design. 

Lets consider it was good enough, if so is this the mistake by Agile vendor ? why couldn't they just develop as per the technical artifacts that came out of inception, why did they had to change while developing ? may be that because they are agile :)

For me it doesn't seem to be a problem with either of the vendors, for the waterfall vendor they prefer upfront design, spec it out and then develop. Any change to that would be a change request. But for the agile vendor, initial discussions are just a guideline and basis for getting started and they prefer to evolve the design while they go.

Now the biggest question is how to bridge these two vendors ? Is it fair to expect the agile vendor to do upfront design just because they are collaborating with the other vendor which is waterfall. Or should the waterfall vendor be more agile while partnering with agile vendor ?? 

At the end its the customer who is loosing money and the quality of the product between these.. its still puzzling me though !!