A post I wrote for the LinkedIn Engineering Blog – http://engineering.linkedin.com/restli/linkedins-restli-moment
Concurrency is a topic that’s close to my heart. It all started when I was introduced to Pthreads in my CS 241 Systems Programming class at UIUC. Having the ability to write code that could be doing multiple things simultaneously was (and is) pretty awesome. The assignments in that class, which included writing a parallel sorting function, a parallel version of
make and a simple HTTP Server, really helped us appreciate the power of concurrency and threading. My love for concurrency has only grown since then and I’ve written concurrent code in both Java and Python.
This Summer I spent some time with Scala Actors and Akka (I got familiar with the concept of actors on Scala 2.8.1 and then upgraded to Scala 2.10.2 for Akka 2.1.4. As mentioned, Scala 2.10 and above will use Akka for actors). I feel that the actor model is a great way to think of and write concurrent programs. By relying only on immutable message passing (Yes, I know that you can pass mutable messages. But please don’t do that.) between actors for communication and coordination I feel that the likelihood of common errors seen in threading based concurrent programs like deadlock, livelock, race conditions etc. are reduced. Moreover, there are certain classes of applications, for example distributed systems applications, that can easily be modeled as a system with entities that communicate by sending messages to each other. Actors would be great for something like this as the actor code would map pretty neatly to the message flows in the system.
Akka builds on top of the actor model and adds a bunch of cool features. Some of my favorite include: supervision and monitoring, being able to refer to any actor using a hierarchical path, FSM support, Netty based remoting, and routers. This is a great example of why I’m a fan of Akka. The author of the post was able to model his problem as an FSM and translate it pretty neatly into code. This example ties in with what I was saying earlier about actors and distributed systems. By decomposing the system into a set of entities and possible messages that can be exchanged between the two the author of the blog post was able to come up with an elegant solution for the problem.
For programmers who’ve only ever used the threading model of concurrency in the past, I say give actors a try! You might be able to come up with a more awesome and well-structured solution to your problem. Akka also has a Java API in case you do not want to use Scala.
We have a function in our Backbone view that calls a function on our Backbone model. The Backbone model talks to our RESTful backend and based on the response triggers the “success” or “error” callback functions provided by the view. Here is the pattern that we used to test this function:
We’ve been using Fabric to set up and build Gelato on AWS. Each time I use it I’m left with this sense of awe at how amazing it is. Going from having to manually SSH into each machine to do anything to have Fabric build your code on 15 machines in parallel is indescribable.
One thing that we were having trouble with was having Fabric run a task on specific host roles in parallel. To run tasks in parallel you use the @parallel decorator, while to run tasks on hosts by roles you use the @roles decorator. If you want to run tasks in parallel on specific hosts you have to be careful of the order in which you apply these decorators. Here is what worked for us:
P.S. make sure you set the correct Bubble Size if you have a large number of hosts!
I’ve been wanting to learn and write something using Redis and Nodejs for the past few days. While scouring the Internet for knowledge I came across this article that describes how to implement a simple pastie.org clone using Redis and nodejs. I u…
I’ve been wanting to learn and write something using Redis and Nodejs for the past few days. While scouring the Internet for knowledge I came across this article that describes how to implement a simple pastie.org clone using Redis and nodejs. I used this article as ‘inspiration’ and modified their sample application.
For the curious, the winners were: 1st prize was the capture the flag game, 2nd prize was taken by Linked Out (an application that used data available on Likedin to predict who will change jobs) and the 3rd prize was grabbed by the Django-streaming-file-sharing application (they called themselves Beamit).
#inday, I shall miss you.
A week or two ago I started learning Django, and wrote my first app, a simple contacts book thingy. Right from the get go, I was amazed by how everything felt so natural in Django, at least to me. The MTV pattern seemed really intuitive and I had …
A week or two ago I started learning Django, and wrote my first app, a simple contacts book thingy. Right from the get go, I was amazed by how everything felt so natural in Django, at least to me. The MTV pattern seemed really intuitive and I had no problem diving right in and creating an app. The Django documentation is extremely well written and answered all the questions I had while coding. Even though I was creating my first Django app, I had no problems in incorporating generic views, model forms, pagination etc. In order to have database migration support (yes, I kept changing the schema even for a simple app :p) I installed South and everything was smooth sailing from there. The last thing I want to add to the app is search capabilities, and for this I’ve decided to use the Haystack application. I’m pretty sure this is overkill for such a simple app, but I wanted to try out this application and hence decided to throw it in.
After working with Django, I’ve decided to go back and give Rails another go. I’ve almost completely forgotten all the concepts from Rails, and I would love to refresh my memory.