Apr 20

So, John brought up a very interesting point in a comment on the first post, which made me think about why I’m doing this monolithically. Truth be told, I couldn’t come up with a good answer.

After making a design diagram and really looking into what I need the backend to do, it became very clear to me that there are three main functions of the backend: collect data, push alerts, and respond to on-demand requests for data.

Items one and two are linked–I need to send a notification out as soon as the data comes in. However, item three can ostensibly exist on it’s own, separate from the other two. As I sat and thought about it, this made more and more sense. When I was first designing the system, WeatherMan had its own protocol for communicating with the server. As I was working, I ended up ditching that for a demand/response system over HTTP, which I kept integrated into the monolithic server.

It’s become apparent to me that this is a waste of resources. I will probably keep the WeatherWire parsing engine/notification system in Java since it’s already written there, but I will be offloading the demand/response system to either PHP or Ruby on Rails, to take advantage of the web server systems that I’m already running anyway.

Yay progress!

Tagged with:
Apr 14

I’ve decided that I’m going to keep doing coding work in Java on the backend, but I am also going to code the same features in parallel in Ruby to learn the syntax and the libraries. I think Ruby’s standard library will take care of most of what I need, and there seems to be a daemonization option out there. If I ultimately feel like Ruby will be the better option I’ll switch the backend over to the Ruby version.

Any Ruby resources you all can recommend?

Tagged with:
Apr 13

So, I wrote a decent backend in Java for WeatherMan last year. While the majority of my frontend code can be preserved, the backend is going to need a lot of work, for several reasons:

  • There was a nasty infinite loop somewhere in the code that I was unable to nail down the root cause of; some condition in the WeatherWire parser was causing the issue
  • I need to write a truly secure communications channel between phone and server; location information is stored and transmitted, and I want to make sure that is secure. Last year I was using a hashed and salted device token to authenticate with the server. However, the communications channel itself was unencrypted, and this required that the device send a token received from the APNS server, which is something I’d like to avoid doing.
  • The backend was never really set up as a daemon; start/stop was manual and there was no external control.

Because of the significant rewrites required, I’m probably going to end up breaking the whole thing down, reusing the code that works, but rethinking the architecture. This got me thinking–is Java really the best language to use for this?

Here is what the backend must do:

  • Keep a consistent connection via telnet to the WeatherWire server to pull down weather alerts
  • Parse the data coming into the telnet connection to decide what is an alert and what isn’t, and to break the alert down into an object that can be stored in the database
  • Keep records of all devices using the software and the locations they have requested to receive alerts for
  • Each time an alert comes in, find the location of that alert, match it to the device database, and send an APNS notification to each device that wants to receive alerts for that location
  • Connect to the NWS FTP servers to download data on-demand
  • Act as a cache for on-demand information (forecasts, radar images) to reduce loads on the NWS servers
  • Insert and retrieve all of this data from a MySQL database
  • Expose an HTTP server over an SSL/TLS connection that will take POST requests for data and return JSON datasets (there shouldn’t be any actual binary data required, as even the radar imagery will be in a KML format that is parsed on the device)

So basically, whatever language is used needs to have good networking libraries and good text/string processing capabilities, and needs to be fast enough to process the WeatherWire data in realtime (there can be quite a bit of it). Also, the backend must be able to run on a Linux server, which immediately counts out Objective-C/Cocoa and .NET for the backend. Here is the list of languages that I’m considering, and what I feel the pros and cons are. Any comments would be appreciated.

  • Java. Via the String class, Java has strong enough text-processing capability to meet my needs. Also, it has a large amount of open-source libraries available; I made extensive use of the Apache Commons libraries in the original version to implement telnet and FTP connections, and used the embeddable Jetty webserver to have a lightweight HTTP server to handle the JSON requests. Plus, while the JVM isn’t as fast as native code, it is generally going to be faster than an interpreter.
  • C/C++. C is a little lower-level than I’d like to use for this application; being able to take advantage of class libraries that will handle the finer points of network connections and string processing is something I would miss dearly. This option would definitely win on speed, however the learning curve and heightened security risks make me wary.
  • PHP. PHP would be the clear winner (at least in my experience) for the text processing and HTTP responses. However, the interpreted nature could cause slowdowns, and while I’ve seen a lot more non-Web platform PHP use recently, I’m not sure if the tools are all there (anybody ever seen a Telnet library written in PHP?). Plus, can it be easily daemonized?
  • Python. I have no experience with Python, but I do know it has a fairly extensive standard library and a lot of developer support. The interpreted nature and potential difficulty daemonizing are a couple of potential pitfalls.
  • Ruby. Another language with fairly extensive developer support that I do not have any experience with. Again with the interpreted nature and daemonization, though based on the Rails framework I could see some ease with the server portions of this.
  • Go. Google’s Go intrigues me as a language. It seems to me that it would offer a lot of the benefits of C/C++ without a lot of the pitfalls (namely direct memory access). The newness of it is a turn-off though–specifically, there is no native connector for MySQL.

If you have any thoughts or insights I’d love to hear them. I’m up for learning something new if it will meet my needs in a better way than Java currently is.

Tagged with:
preload preload preload