Monday, June 6, 2011

Using The Dojo Build System To Speed Up Your ESRI JavaScript API Apps

[Updated 4-15-2013]
I've come up with a solution for ESRI JSAPI 3.4 and AMD.


As your JavaScript projects get more and more complex, loading all of those Dojo classes can really slow down your load time. All those dojo.require calls add up in a hurry. The Dojo Build System can be a huge help in speeding up the load time and general performance of your apps. For example, a build that I ran on a recent project took the number of JavaScript requests on page load from 53 down to 5. The css request went from 16 down to 4. This ended up cutting the load time in half! Other nice features include stripping out all of the console calls, minifying your JavaScript and interning all of your widget templates.

The rest of this post assumes some familiarity with the Dojo Build System. If you haven't looked at it before, the documentation is worth reading. There's even a fancy new tutorial.

After reading all of the Dojo documentation it's easy to get excited about the possibilities. However, you will quickly find that mixing the ESRI api into the equation makes a big mess of everything. For example, the dojo build system assumes that you are hosting everything yourself. But because ESRI has not released a source/unbuilt version of their api that we can download we are stuck loading Dojo from their servers. The other problem is that when you load the ESRI api you are really loading their layer file which can have a lot of overlap with your layer file thus adding a lot of duplicate code. Not to mention the problems that the build system has when it sees: dojo.require("esri..."); and it doesn't know where to get it. Over the last few months I've developed a solution to overcome these problems and end up with a lean and mean (for the most part) product in the end.


First, let's look at my project template. You can check it out on GitHub here. I keep separate folders for the source version ("src") and built version ("build"). This makes it easy to update our servers when I do a new build. I just checkout the "build" folder through SVN and update when needed. Something not shown is a local copy of the Dojo source code that I have downloaded. This contains the Dojo Build System (see "util\buildscripts"). I keep only one copy of the Dojo code on my local machine and use it for all of my projects. This is possible because I keep my build profile within a project folder in "buildsupportfiles". That way I never touch the Dojo source code which makes it easier to upgrade to new versions.


Now let's take a look at my build profile template ("build.profile.js"):

dependencies = { cssOptimize: "comments", optimize: "shrinksafe", layerOptimize: "shrinksafe", action: "clean,release", version: "1.6.1src", internStrings: true, mini: true, stripConsole: "all", releaseName: "PROJECTNAME/content", layers: [ { name: "esriapi.discard", resourceName: "esriapi.discard", discard: true, dependencies: [ "dijit.WidgetSet", "dojo.fx.Toggler", "dojo.Stateful", "dijit._WidgetBase", "dijit._Widget", "dijit._Templated", "dijit._Container", "dijit._CssStateMixin", "dijit.form._FormWidget", "dijit.form._FormValueWidget", "dojo.dnd.Mover", "dojo.dnd.Moveable", "dojo.dnd.move.constrainedMoveable", "dojo.dnd.move.boxConstrainedMoveable", "dojo.dnd.move.parentConstrainedMoveable", "dijit._HasDropDown", "dijit.form.Button", "dijit.form.DropDownButton", "dijit.form.ComboButton", "dijit.form.ToggleButton", "dijit.form.HorizontalSlider", "dijit.form._SliderMover", "dijit.form.VerticalSlider", "dijit.form.HorizontalRule", "dijit.form.VerticalRule", "dijit.form.HorizontalRuleLabels", "dijit.form.VerticalRuleLabels" ] }, { name: "../PROJECTNAMELyr.js", resourceName: "PROJECTNAMELyr", layerDependencies: [ "esriapi.discard" ], dependencies: [ "js.core" ] } ], prefixes: [ ["agrc", "../../../../REST/PROJECTNAME/src/content/agrc"], ["css", "../../../../REST/PROJECTNAME/src/content/css"], ["html", "../../../../REST/PROJECTNAME/src/content/html"], ["ijit", "../../../../REST/PROJECTNAME/src/content/ijit"], ["images", "../../../../REST/PROJECTNAME/src/content/images"], ["js", "../../../../REST/PROJECTNAME/src/content/js"] ] }


The end goal of this profile (other than some optimized css files) is the "PROJECTNAMELyr.js" file. "js.core" is my main JavaScript file for the project and includes all of the necessary dojo.require(...) calls. The build system takes that file and combines all of the dojo classes that are referenced and rolls them up into one compressed file. This means that the browser only has to make one request and parse a single file.

I did encounter a problem the first time that I tried a build that included a dojo.require call for some esri classes (ie. dojo.require("esri.dijit.Legend");). Since I don't have a local, unbuilt version of the ESRI API, the build script chokes because it can't find the appropriate files. After a little digging around I was able to find a work-around. If you use dojo["require"]("esri.dijit.Legend"); then the build script will skip it and leave it alone. So the file does not get rolled into your layer file, but a least it can finish your build. Then when the layer file loads it will still load the ESRI classes via dojo.require.


The "esriapi.discard" layer makes sure that I don't include any dojo classes in my layer file that are already in ESRI's layer file. I found this list of classes by searching their layer file for "dojo.declare".

After my build is finished, I delete the dojo and dijit folders (those will be loaded via ESRI's servers), and copy everything else to my "build/content" directory. The only thing left to do is re-point my web page to load the layer file.

Sometimes after I re-point my page to the layer file, I get the following error message: "uncaught exception: Could not load cross-domain resources: dojo.nls._en-us". The nls is the localization stuff. Even though I do copy that folder into project, for some reason I still have a problem. I have a feeling that this is related to the fact that I'm loading my layer file from a different domain that Dojo is being loaded from. The office genius (@SteveAGRC) helped me with a solution for this problem. If you load the _en-us file manually before you load the layer file, then you don't get the error message. So my header looks like this:

<script type="text/javascript" src="http://serverapi.arcgisonline.com/jsapi/arcgis/?v=2.3"></script> <script type="text/javascript" src="content/nls/PROJECTNAMELyr_en-us.js"></script> <script type="text/javascript" src="content/PROJECTNAMELyr.js"></script>


In the end, the Dojo Build System has cut my page load times down significantly thus giving my users a better experience. If you are headed down a similar road I hope that this post can save you some time beating your head against the wall.

Another helpful post:
http://blog.geocortex.com/2009/05/12/build-your-jsapi-applications-for-performance/

25 comments:

  1. Thanks for this explanation. I have a few questions.

    Your end product is the custom layer file, a single javascript file. Am I correct that this file includes only dojo code and no ESRI code? In other words, the entire process you described above does nothing to streamline loading ESRI-declared widgets/classes?

    Next question. When you say you searched the ESRI layer file for all their dojo.require statements, which file is that?

    ReplyDelete
    Replies
    1. Yes, at the end of this process I have a single javascript file with no ESRI code. I still have to load the ESRI code separately. That also means that any ESRI classes that are not in their layer file get loaded via dojo.require. I spoke with them about this at the dev summit a few weeks ago and they said that with Dojo 1.7 and AMD there would be other options for this problem. Possibly a custom builder for their stuff.

      The ESRI layer file is the file that you get when you load: http://serverapi.arcgisonline.com/jsapi/arcgis/?v=2.8

      I hope that this helps.

      Delete
  2. Thanks for the reply. Couple more questions. You state,

    "The "esriapi.discard" layer makes sure that I don't include any dojo classes in my layer file that are already in ESRI's layer file. I found this list of classes by searching their layer file for "dojo.require"."

    If I search the layer file you pointed me to above for dojo.require statements, I do not see the dojo require statements you listed above. I only find these two.

    dojo.require("dojo._firebug.firebug");
    dojo.require("dojo._base._loader.loader_debug");

    Next question. Do you create a cross domain build/layer file? Would you post your command line statement for the build?

    ReplyDelete
    Replies
    1. Yep. I meant to say that I searched for dojo.declare statements in the ESRI layer file.

      Creating a cross domain build is as easy as adding an additional property to the build profile, "loader: xdomain". You can go here for more info: http://dojotoolkit.org/reference-guide/1.7/build/xDomain.html

      This is what I use to kick off my builds: https://github.com/stdavis/AGRCJavaScriptProjectBoilerPlate/blob/master/buildsupportfiles/build.bat

      Delete
  3. hi.. I am currently trying create a build of my custom package using dojo 1.7 build system(as a custom layer). I have a couple of esri modules which are used as part of one of my class. (This class declaration uses AMD style (i.e) define instead of require statements). Since I don't have an unbuilt version of esri api, I am unable to get the build working without commenting out both my esri defines. Any ideas how I can circumvent the problem in dojo build system 1.7.

    ReplyDelete
    Replies
    1. I haven't used 1.7 yet since esri is still at 1.6.1. You may have to wait until esri changes their module loading to AMD and upgrades to 1.7. Please let me know if you get it figured out.

      Delete
  4. This links doesn't work

    https://github.com/stdavis/AGRCJavaScriptProjectBoilerPlate/blob/master/buildsupportfiles/build.bat

    ???

    ReplyDelete
    Replies
    1. Sorry about that. I moved the repo. I've fixed the links in the article now.

      Delete
  5. thanks for fixing the link. I tried your method - seems to work ok with 2.8 of ags javascript api :).

    Now that esri use 1.7, have you got this method working with ags javascript api 3.0?

    ReplyDelete
    Replies
    1. I'm currently working on updating our build process for the esri api 3.0 and dojo 1.7. I'll publish a new post when I'm finished and will link to it from this post.

      I'm hoping with AMD that I will be able to build ESRI's api into my layer file and won't have to load it separately.

      Delete
  6. I have some issues with this when trying to create a xdomain build that includes a dijit that creates an instance of esri.toolbars.edit?

    ReplyDelete
    Replies
    1. Are you using dojo["require"]('eseri.toolbars.edit') instead of dojo.require('esri.toolbars.edit')?

      Delete
  7. Yes, we tried using dojo["require"]('eseri.toolbars.edit') within the dijit to do the xdomain build. When running the application we get an error that indicates it is unable to load "_graphicMover". Everything else seems to work ok. The same dijit file work ok locally.

    ReplyDelete
    Replies
    1. Not sure what's going on then. Other than you misspelled 'esri' but maybe that's just a typo in your comment above and not in your code.

      Delete
  8. Great article, many thanks for the explanation. I've been trying to get to the bottom of the Dojo system for a while.

    ReplyDelete
  9. Has anyone tried to do a build for 3.4, I'm keen to understand how to .... so i can publish up a hybrid application in the mobile space.

    ReplyDelete
    Replies
    1. I'm about to try. I'll post my results. A little worried about how to deal with ESRI AMD imports since the build system will not be able to access them.

      Delete
    2. Hi Scott,

      I am still struggling to migrate dojo build system written for ArcGISv2.7 to v3.8. Do you have any better suggestions for this?

      Delete
  10. Hi Scott, How did you manage to write the dojo build system for ArcGIS javascript versions 3.4 +..

    ReplyDelete
    Replies
    1. http://geospatialscott.blogspot.com/2013/09/the-esri-api-for-javascriptdojo-build.html

      Delete
    2. Thanks a lot Scott, Actually we are hosting ArcGIS JavaScript library locally and dojo build has become more painful since it goes on including all the esri files(and their internal references) into build. I think your 'AGRC's JavaScript Project Boilerplate' may help me in this regard. Do you have any suggestions for me?

      Delete
    3. Yes, if you use the build system in the boilerplate you do not need to load the esri api from their servers in the built application.

      Delete
    4. Thanks a lot Scott, will try this now

      Delete