When I undertook my first "big" web application in 2009, I did it in Django. The project was fairly ambitious not just because it was a web application, but it was also my first foray into cryptographic protocols. I was trying to design a system where the client and the server didn't necessarily trust each other.
Note
As an exercise for any programmer who hasn't delved into cryptography, I have a suggestion. Pick an application you've written and look at a communication, then ask: "What if this piece didn't trust that piece?" We often take a lot for granted, and seeing things in this new light really helps you appreciate how nuanced a cryptographic protocol can be. I'd suggest reading through Applied Cryptography if you haven't--I did before starting my project, and it was very accessible and an interesting read.
In any case, when you've got two pieces of a system that need to double-check each other, you wind up running the same computations twice. And in this case, I had to write two copies of the code in different languages...once in JavaScript, and once in Python. More generally, I was frustrated with having to flip mindsets between one language on the server and another in the browser.
This led me to go around in a rant to people which sounded something like this:
Argh. Python, JavaScript, Ruby...who cares! They're all basically the same. I probably like Ruby the most and JavaScript the least, but the essential differences are pretty slight. Any perceived "betterness" is simply outweighed by the disadvantage of having to use more than one language. Given that the browser isn't going to change, I'll bet you that in the end people are just going to start doing server-side JavaScript.
There were varying reactions to these rants. Some people nodded in agreement. Others argued that JavaScript was too lame to ever make inroads on the server. But occasionally someone would say "y'know, there are actually people who are doing exactly what you describe". So they'd send me some server-side JavaScript page (and I'm sure Node.JS was one of them). I'd look at the scrappy pages and go "yup, something like that"...but I never installed any of it. I was more interested in being a prophet about what was going to happen than investing even more heavily into a language I didn't really like in the first place.
But in 2012 I went to a virtualization conference hosted by VMWare. I showed the demo of my Python-server-based system to one of the guys from Cloud Foundry. He thought the project was cool, but mentioned they didn't have Django support, and suggested I port it to Node.JS. There were representatives from MongoDB there too, which was one of the facilities that came on Cloud Foundry. It looked like it wouldn't be too hard to switch from Django/MySQL to Node/MongoDB for the server component, and things might simplify a bit.
So I decided maybe the time had come to grit my teeth and embrace my prophecy. The result is what is now published as Blackhighlighter.
Well...I was half right
Node.JS has taken off in popularity, which shows that I was onto something. Yet in my case I wanted to just share some simple routines between the browser and the server. This turned out to not be straightforward at all, despite seeming like one of the most basic things you should be able to do. In fact, there aren't that many people who really care about this.
The "module system" in Node.JS is pretty reasonable. It's managed by a tool called "NPM", and if your node program has dependencies you just put them into a
package.json file with a version indicator you want. You can use wildcards too, so asking for 0.10.* will get you the latest release of subversion 10 of something.
But for those of you who don't know, the JavaScript world is a hornet's nest of oddities, workarounds, byzantine libraries, competing approaches, and weird and tiresome arguments.
The core of the problem is that when web browsers were released, they just lumped everything global into a single object, called window. This resulted in some workarounds that are known as the "module pattern", and you can read it explained and what's considered "best practices". But when Node.JS was released, if you declare a file-global element, it isn't seen by other files in the project.
In the browser, a global declaration implicitly becomes a member of the
window object. Node's parallel to window is process (from which you can get things like environment variables, and lots of other things). But a key difference is that an unqualified global declaration does not implicitly become a member of process; it is only available inside that module unless you explicitly stick it into an exports object.Note
Not only aren't globals in one module not seen by default, there seems to be no way to work around it. In digging a little into the V8 JavaScript engine source code and looking how functions were declared, I saw that a "function definition" doesn't get added to the current context. Instead, they're put into an "activation object" of the "execution context"; there doesn't appear to be any kind of back door in to let you see inside such things from another module, unless you're using some kind of debugger.
Meet RequireJS and Browserify
As we all know, to use a JavaScript file in a web page you use a <script> tag. All of the scripts are loaded asynchronously as separate HTTP requests, and getting a "moment of synchronization" is one of those apparently-not-thought-about-beforehand-but-is-obviously-neessary things that people had to work out after the fact.
Node.js comes with a loading function called "require" which synchronously loads script code from the local disk. It has some amount of convention so you don't have to necessarily specify the full path to a module, and you also omit the ".js" extension. As this is the only way, it means that if you are looking to share code between client and server code with node.js, you need some kind of adapter.
The most thoroughly-considered adapter is named RequireJS. It introduces an inclusive standard that means you write your modules in a way that isn't compatible with either require or the <script> method (facepalm).
Long story short: you can use
require ordinarily in the node modules, unless that module is intended to be shared between the browser and server. If you want to include one of those shared modules, then use the the keyword requirejs to import it.
Take three clicks in any direction, and you will land in a passionate arguments from people who say that the better alternative is to introduce a compilation step. Basically throw asynchronous loading out the window, and glom everything needed by your page into one file to blob over the network. Your node program won't care as the files are local, and the browser would probably rather just have everything sent as a minified blob anyway. How many pages do anything meaningful when they're "half loaded", anyhow? A popular choice in this area is called Browserify
But if you value things like...oh, debugging, say...or if you're explicitly trying to convince people your code has nothing to hide so they trust it, you may not want to obfuscate like that. So I went with RequireJS. Having thought about the art of software engineering a lot, I cry a little inside when I think that this is what young programmers are reading up on today and consider it "learning something". HOWEVER--it works, is documented and supported, and there are enough people using it that someone will presumably write the "how to change your code from requirejs to the real answer".
Note
That is, to the extent that the JavaScript ecosystem ever has had, or will have, "real answers". :-/ Sorry folks, Node/V8 and the debugging tools in browsers like Chrome/Firefox etc. may be JavaScript's 15 minutes of fame. It's best hope is to become an invisible underpinning like X86, which engineers curse every day, while working around it in their processor design by translating into something decent prior to execution.
jQuery, Underscore.JS, or native JS in shared code?
BlackHighlighter is a jQuery plug-in which needs to share code between the client and the Node.JS server. A key constraint on it is that I did not want to include jQuery on the server.
There actually is an NPM jQuery package, and you can do DOM queries with it. If you mention it people will groan and ask why you would want such a thing, and admonish you not to use it. I see their point--and preferred to use underscore.JS on the server side. Yet I didn't want to depend on underscore AND jQuery on the client. This put me in a bit of a bind.
You can read the blackhighlighter source. But the punch line is that I require the jQuery widget source on the server side, but shim it with a "fake" jQuery. The shared code uses native JavaScript calls. This cuts down the dependencies and so far it has worked out pretty well. Details are in this StackOverflow question:
