grunt-esri-slurp - Make Your Own ESRI JS Package

I recently contributed to a blog post about a great tool for scraping ESRI's AMD build of their JS API. If you are interested in doing your own builds with the Dojo Build System and ESRI's JS API, you should definitely check it out.

Demystifying the Dojo Build System - 2014 Dev Summit Presentation

The ESRI Dev Summit this year was awesome as usual. This was my third year and it keeps getting better and better for me. I love being able to have direct access to ESRI developers and rubbing shoulders with amazing developers working on the same problems that I am. And the plenary this year was really great.

I was privileged to present again this year. My submission title was Demystifying the Dojo Build System. Here's the abstract:

If you are not using some sort of build system for your JavaScript apps, then you are missing out on some huge performance gains. Concatenation, minification, and interning strings will almost certainly shave seconds off of your page load times.The Dojo Build System is a program that can apply these types of "deployment optimizations" to your source code. However, it can be a steep learning curve and throwing the ArcGIS API for JavaScript into the mix only complicates the situation. This presentation will untangle the build system and give you a solid overview of all of the moving parts. We will explore real world examples of how the Utah AGRC uses this system in our web applications and how it can be applied to your applications as well.

It obviously struck a cord because the room was packed. My presentation ended up turning out well and I got some great feedback. I think that there are a lot of people interested in building their applications but the Dojo Build System can be intimidating (see my first slide below).

I was concerned that it would be overshadowed or rendered irrelevant by the new web optimizer that ESRI is releasing shortly and previewed at the conference. However, this was not the case. The web optimizer looks awesome and will be a huge help for a lot of people but there will always be those that want to keep the build process local and want total control over it. Hopefully my presentation will save these people some head aches.

Here's some resources from my presentation:

Summary Sheet
Example Projects

Video

Slides

The ESRI API for JavaScript/Dojo Build System Saga Continues...

That's right, my entire app JS is in this one file *including @ESRI modules* http://t.co/06S4KCtkuR cc @DavidSpriggs @derekswingley
— Scott Davis (@ScottAGRC) May 21, 2013

This is my third post on this topic. First, there was discard layers. Then AMD threw a curve ball. And now a little bird told me that, starting at version 3.4, there is a special build of ESRI's API for JavaScript that is straight up minified AMD modules with no layers files. I'm not sure why this hasn't been more publicized. It's just a little minification away from releasing their source coded. And, as I will outline below, it allows you to build an honest-to-goodness layer file without any of the superfluous code that past versions required. This means that you can build all of the JavaScript code for your application including Dojo, ESRI and your own modules into a single, minified file.

To load the new AMD build of the ESRI API for JavaScript you just append "amd" onto the end of the url. For example, js.arcgis.com/3.6amd (3.5 requires something different because something is messed up with their build: http://js.arcgis.com/3.5amd/js/dojo/dojo/dojo.js). You should be able to just make this one change in your app and it should still work. You may need to explicitly require a few more ESRI modules now that you are not getting a bunch up front with their layer file. With this build you have to load each module individually. So it's going to slow down your load time significantly. It took our boilerplate from 128 requests to 337 requests to load the unbuilt, source version of the app. Needless to say that you should never use this url in production! In fact, I don't even use it during development because it takes so long to load.

The real usefulness in the new AMD build happens when you feed it through the Dojo Build System. The first step is to scrape it to your machine. In order to do this, I needed a list of all of the ESRI modules and other resources like images and .html files. I started with their argument aliases doc and then filled in the gaps by hand. It was easy to see what files I was missing since the build system returns errors for missing dependencies. After I had a good list of files, I used a bash script to pull the files down to my local machine. Here's the script:

The files cannot be fed directly into the build system as is. You'll notice that, in the script above, I had to fix some artifacts from their build (e.g. define('dep1,dep2,dep3'.split(','))...) as well as some paths to the dojo directories. After running this script you have the entire ESRI JavaScript API as separate modules that are ready for the build system. This is so ridiculously close to having the actual source code (just a bit of minification), I wonder if they are just going to release it in the future.

After getting the ESRI modules onto your computer, it's just a matter of pointing to them in your build profile and you are good to go.

You can see the entire set up for this process in AGRC's JavaScript Project Boilerplate. Hopefully the next blog post is about how awesome it is that ESRI has finally released their source code. ;)

Using The Dojo Build System To Speed Up Your ESRI JavaScript API Apps

[Updated 4-15-2013]
I've come up with a solution for ESRI JSAPI 3.4 and AMD.


As your JavaScript projects get more and more complex, loading all of those Dojo classes can really slow down your load time. All those dojo.require calls add up in a hurry. The Dojo Build System can be a huge help in speeding up the load time and general performance of your apps. For example, a build that I ran on a recent project took the number of JavaScript requests on page load from 53 down to 5. The css request went from 16 down to 4. This ended up cutting the load time in half! Other nice features include stripping out all of the console calls, minifying your JavaScript and interning all of your widget templates.

The rest of this post assumes some familiarity with the Dojo Build System. If you haven't looked at it before, the documentation is worth reading. There's even a fancy new tutorial.

After reading all of the Dojo documentation it's easy to get excited about the possibilities. However, you will quickly find that mixing the ESRI api into the equation makes a big mess of everything. For example, the dojo build system assumes that you are hosting everything yourself. But because ESRI has not released a source/unbuilt version of their api that we can download we are stuck loading Dojo from their servers. The other problem is that when you load the ESRI api you are really loading their layer file which can have a lot of overlap with your layer file thus adding a lot of duplicate code. Not to mention the problems that the build system has when it sees: dojo.require("esri..."); and it doesn't know where to get it. Over the last few months I've developed a solution to overcome these problems and end up with a lean and mean (for the most part) product in the end.