Monday, April 6, 2015

Windows Scheduler: Get Your Priorities Straight

At AGRC we have a variety of tasks (usually python scripts) that need to be run on a schedule. These are usually workflows that scrape and ETL data for web applications. Currently we use Windows Scheduler to run these scripts. Recently I've had problems with scripts taking way too long to complete. After a bit of digging I discovered that, by default, Windows Scheduler assigns a process priority of "Below Normal" to all tasks. The pain point is that they provide no UI to change this setting. After a bit of digging I found the following steps to work around this problem by hand editing the xml export of a task.
  1. Right-click on the task and export it as an xml file. 
  2. Open the file in a text editor and search for the "Priority" element. 
  3. Change the value of the this element to the desired priority level. See this page for a list of possible values. Usually 6 is what you want. 
  4. Save your changes and close the xml file. 
  5. Delete the original task and re-import the modified xml file as a new task. 
More details

Sunday, April 5, 2015

I'm Changing My Twitter Handle

For me Twitter has been almost exclusively a tool that I use in my profession to connect with others doing the same work as me and to keep up with the ever-changing landscape around geospatial software development and web development in general. I've never tweeted about what I was having for breakfast or what my crazy five kids are up to.

Twitter has been an invaluable resource for me and I work hard at doing my little part to keep the service useful and relevant. I quickly unfollow people who let their streams wander into meaningless or offensive paths that at best waste my time and at worst degrade my mind. My time is incredibly valuable to me and I try to treat others with the same attitude.

Occasionally, I feel compelled to share sometime that's not related to work but nevertheless valuable, uplifting or inspiring. It might be related to my favorite hobby, parenting or my faith. I've largely kept these to myself. In part because my twitter handle included the name of my employer and also in part because I didn't know if it would be appropriate.

However, over the past few years, I've felt that I should do more to counter the indecency of the internet conversation. In the midst of so much darkness I feel like the light that I have to share can make a real difference, that I can do a bit more to keep my tiny part of the internet out of the shadows.

For these reasons I've decided to change my twitter handle from ScottAGRC to SThomasDavis to allow me to feel more free in what I share. I don't foresee a huge change in my stream. But don't be surprised to see a bit more faith-based content. In sharing I don't mean to offend and would be happy to answer any honest questions that may arise.

Happy Easter!

Tuesday, January 20, 2015

How To Use AGRC Base Maps in QGIS

Most people know about AGRC's awesome base maps. They are very popular and provide high quality cartography using the latest and greatest data from the Utah SGID. But did you know that they provide a WMTS service that can be consumed in non-ESRI products?
Here's how to load our base maps in QGIS 2.6.1:
  1. The first step is to find the URL to the service that you are interested in. Most of AGRC's base maps are within a folder called "BaseMaps" on our main ArcGIS Server instance. Once you find the specific layer that you are interested in, copy the URL for the WMTS link at the top of the services directory page:
    link
  2. Open QGIS and click on the "Add WMS/WMST Layer" button to open the "Add Layer(s) from a WM(T)S Server".
  3. Click on the "New" button to open the "Create a new WCS connection" dialog and add a name for the layer and the URL to the WMTS service and click "OK" to close the dialog.
    dialog
  4. You should now see a new layer in the add layer dialog. Select the new layer and click on the "Add" button to add it to the map.
    dialog
  5. You should now be able to view the base map as a layer in QGIS!
    map

Bonus Tip

If you are having performance issues using our cached services through ArcMap, try loading them via these WMTS services. You can do this by double-clicking on the "Add WMTS Server" node in the ArcCatalog tree under "GIS Servers" and then pasting the same URL as above. arcmap

Tuesday, November 18, 2014

How to Wire up Travis-CI to your JS Projects

For the past six months, AGRC has been using Travis CI to automatically test and lint our projects each time we push a commit to the associated GitHub repository. Even though we run these tasks locally it's been helpful to have them run on Travis for when we miss things. It's also a major step towards automated deployments as well as running our tests via something like Sauce Labs or Browser Stack.

travis-ci.org

The set up is relatively simple. The first step is to sign into travis-ci-.org with your GitHub account. Once you're signed in you can go to your accounts page and see all of the GitHub repositories associated with your account. Switching a repository to "ON" tells Travis-CI to start watching any new commits that you push to that repository.
accounts page

.travis.yml

The next step is to let Travis-CI know what you want it to do. The first part of this step is accomplished by creating a .travis.yml file at the root of your project. Here's an example from one of our projects:
language: node_js
node_js:
  - '0.10'
before_install:
  - npm install -g grunt-cli
  - npm install -g bower
  - npm install
  - bower install
notifications:
  email:
    on_success: never
The lines below before_install load all of the project dependencies via npm & Bower. The notifications code just tells Travis to only send us emails when a build fails.

package.json

The second part to defining what you want Travis-CI to do is to add a scripts property to the your package.json file for your project. Travis-CI automatically runs npm test for NodeJS projects. Adding this new property to package.json defines this command. We use a special travis GruntJS task to run tasks so this is the command for us:
"scripts": {
    "test": "grunt travis -v"
}
The travis grunt task can contain any sub-tasks that you want. Here's what ours looks like:
grunt.registerTask('travis', [
    'if-missing:esri_slurp:travis',
    'jshint',
    'connect',
    'jasmine:app',
    'build-prod'
]);

Build Status Badge

The icing on the cake is to copy code from Travis-CI to your app's README.md to show a "build:passing" or "build:failing" (gasp!) badge. You can do this by going to your project's page on travis-ci.org and clicking on the badge in the upper right-hand corner of the page.

GitHub.com Integration

After getting everything wired up you'll notice that pull requests automatically display the build status of each commit and will let you know if it is still waiting on a build to run.
still waiting
still waiting
good to go
good to go
If you want to see all of this in action you can checkout the AGRCJavaScriptProjectBoilerPlate repository.

Monday, September 22, 2014

grunt-esri-slurp - Make Your Own ESRI JS Package

I recently contributed to a blog post about a great tool for scraping ESRI's AMD build of their JS API. If you are interested in doing your own builds with the Dojo Build System and ESRI's JS API, you should definitely check it out.

Friday, March 28, 2014

Demystifying the Dojo Build System - 2014 Dev Summit Presentation

The ESRI Dev Summit this year was awesome as usual. This was my third year and it keeps getting better and better for me. I love being able to have direct access to ESRI developers and rubbing shoulders with amazing developers working on the same problems that I am. And the plenary this year was really great.

I was privileged to present again this year. My submission title was Demystifying the Dojo Build System. Here's the abstract:
If you are not using some sort of build system for your JavaScript apps, then you are missing out on some huge performance gains. Concatenation, minification, and interning strings will almost certainly shave seconds off of your page load times.The Dojo Build System is a program that can apply these types of "deployment optimizations" to your source code. However, it can be a steep learning curve and throwing the ArcGIS API for JavaScript into the mix only complicates the situation. This presentation will untangle the build system and give you a solid overview of all of the moving parts. We will explore real world examples of how the Utah AGRC uses this system in our web applications and how it can be applied to your applications as well.
It obviously struck a cord because the room was packed. My presentation ended up turning out well and I got some great feedback. I think that there are a lot of people interested in building their applications but the Dojo Build System can be intimidating (see my first slide below).

I was concerned that it would be overshadowed or rendered irrelevant by the new web optimizer that ESRI is releasing shortly and previewed at the conference. However, this was not the case. The web optimizer looks awesome and will be a huge help for a lot of people but there will always be those that want to keep the build process local and want total control over it. Hopefully my presentation will save these people some head aches.

Here's some resources from my presentation:

Summary Sheet
Example Projects

Video

Slides

Wednesday, January 29, 2014

My Favorite Sublime Text 3 Plugins & Configs

I'm a huge fan of Sublime Text 3. I love it's simplicity and strong package community. Here's a list of my favorite packages and config tweaks:

Packages

Package Control - This enables you to easily search for and install packages. This is always the first thing that I do with a new install of Sublime.
AdvancedNewFile - Best way to create new files.
Auto Semi-colon - Add a semi-colon to the end of a line even if your cursor isn't at the end of the line.
All Autocomplete - Adds autocomplete for words found in all open files. Works well as a supplement to SublimeCodeIntel
Emmet - Awesome shorthand for creating HTML markup. As an added bonus it forces you to learn to write CSS selectors better.
GitGutter - See which lines have been changed since your last commit.
Markdown Preview - Allows your to preview markdown docs in your browser.
TrailingSpaces - Kill all trailing spaces in the current file.
Sublime Code Intel - Auto complete awesomeness
Sublime Linter - Lints all sorts of languages. I use it to auto-run jsHint on my files. So useful to see lint errors as I'm coding.
Sublime-text-git - Have to manually clone and checkout python3 branch to get this to work in sublime text 3. It's a bit of a pain but worth it.

Themes

http://colorsublime.com/
Argonaut
Flatland
Soda
Spacegray
Font: Source Code Pro

Config

Notice that Vintage does not show up in my ignored_packages and I even have it default to start in vintage command mode. I made this switch about a year ago and have never looked back. I really feel like I am more efficient getting around and editing code.

I've had a few people ask me about my Sublime setup and wanted to get it out there. Any cool stuff that I'm missing?