Recently I've been researching a mobile project that will require offline-capable base maps. After unsuccessfully finding a solution already built I decided to try wire together Phonegap, Leaflet, and TileMill's .mbtiles. After a few late nights I was able to see it come together and sent out a quick tweet. This has generated quite a few requests for more information so I threw up a repo on GitHub to demo what I was able put together.
The main idea is to download the .mbtiles to the device using Phonegap's File API. Since .mbtiles are just sqlite databases I was able to use a SQLitePlugin to open them up (thanks, coomsie!). I did run into a problem getting this plugin to read BLOB fields from the .mbtiles database. However, after a bit of poking I was able to get it to work.
Once I had access to the encoded image data it was only a matter of writing a custom leaflet tile layer (TileLayer.MBTiles.js) inspired by a similar one that coomsie had done. One of the big secrets was passing
{scheme: 'tms'} into the constructor.Initially the performance was quite good on the iPad 2 and the iPhone 4 that I tried it out on. However, after cleaning up the project in preparation for uploading it to Github, the performance has suffered a bit. Not sure what I did, but I'll post an update when I get it figured out.
This is my first experience with Phonegap and Objective-C so any suggestions for improvements would be greatly appreciated.
[Added on 4-26-12]
P.S. The app downloads the .mbtiles file automatically from my dropbox account the first time that it runs and stores it in the Documents folder locally. Each subsequent time that it runs it uses this local file. So after the initial download you should be able to open up the app and see tiles while in air-plane mode.
Cool, I've been wanting to do the same thing! Don't hesitate to put up more info.
ReplyDeleteIt would be cool to get this to run on android as well, but I guess one would need another sqlite plugin (plus, leaflet and android don't like each other that much at this point, it seems).
Yes, getting it to work on Android is my next step. I plan to check out this for a phonegap sqlite plugin: https://github.com/chbrody/Cordova-SQLitePlugin
DeleteHi Scott,
DeleteDo you still plan to get this work on Android? I've used chbrody's Cordova-SQLitePlugin (instead of coomsie's but I had to add some code from coomsie's) to have your code working and running on my iPad with Cordova v1.8.1, but haven't tested it on Android. I'm hoping it doesn't require too much changes.
Could you please let me know if you have made it work on Android or if you have any suggestions? Thanks so much in advance.
Yvette,
DeleteWe did get it working in Android. However, it was done as part of a project under contract so we can't share the code. The iOS version that I reference in this post was something that I did on my own before we had a contract signed. I did this so that I could share it. :)
Thanks Scott!
DeleteI appreciate the information. I will try to figure it out on my own. :)
Hi Scott...thanks for the post. Really helped me out and works well (on Cordova-1.9.0.js). I've got one problem though!
ReplyDeleteWhen extracting the tile images from the local sqlite db (in the L.TileLayer extension), I get the right tile data as a JSON object but cannot seem to turn the tile_data blob back into a real image (this is based on Android using the Cordova-SQLitePlugin code). Seems to make an invalid image file. I've not tried the iOS version (yet!).
Could this be related to the Android SQLitePlugin code for reading the blob data out of the DB (similar to the issue you had a in #3)? Any thoughts?
Laine
Actually scratch my last question - fixed now (I wasn't decoding the blobs coming back from the DB correctly, now encoding them to base64 strings and all works correctly).
ReplyDeleteGlad to hear that you got it figured out.
DeleteHi Laine,
DeleteCould you please post your fix here or post your project on Github? I'm going to try the code on Android and it would really help me out.
Thanks in advance!
Hello Scott... i tried your repo it is running fine but not showing the map just only the leaflet frame with a zoomin zoomout controle. what could be the possible problem can you help me
ReplyDeleteSounds like there is something wrong with your database. You might want to add some log statements to make sure that you are getting the base64 encoded data from the plugin successfully.
DeleteHello Scott how are you! Thanks for reply
ReplyDeleteI tried some alerts in plugin.js file but not every alert appears
this.openSuccess || (this.openSuccess = function() {
alert("DB opened: " + dbPath);
});
this.openError || (this.openError = function(e) {
alert(e.message);
});
these alerts never appears neither the alert inside SQLitePluginTransaction.prototype.complete.....
I also applied break points in objective c plugin files but only
-(CDVPlugin*) initWithWebView:(UIWebView*)theWebView
function is called out...
In Xcode console i am getting the following log
-[AppDelegate getCommandInstance:]: unrecognized selector sent to instance 0x913fba0
*** WebKit discarded an uncaught exception in the webView:decidePolicyForNavigationAction:request:frame:decisionListener: delegate: -[AppDelegate getCommandInstance:]: unrecognized selector sent to instance 0x913fba0
I am too dumb to solve this problem please suggest me a solution
Thanks for your help
Sounds like you need to go back in the process even further and confirm that the .mbtiles file was downloaded successfully.
DeleteI re-download the mbtiles files from your dropbox link but still it is just showing the leaflet frame, though i am not able to see the map but when i drag the map[invisible map :(] or zoom in or out its making the calls and i am getting the alerts.. may be these alerts are of loading the tiles cause number of times same alert appears is different....
ReplyDeleteThanks appreciate your support
Just in case.. I using cordova 1.9.0
DeleteNeev,
DeleteWere you able to iron this out, I am getting the same error, or issue.
Shane
I realized that the Plugin was not working right the way I was using the build.phonegap.com, looks like I need to test it via x-code.
DeleteNot really sure what's going on. I would try and log the output from the sqlite plugin and make sure that it's getting the encoded data. Then you could paste it into an online service that will translate it into an image to make sure that it's getting good images. Also, make sure that the map div is visible when you are creating it.
ReplyDeleteGotcha, It looks like the sqlPlugin ins the one that isn't working, if I do this:
ReplyDeletedb.executeSql("SELECT tile_data FROM images INNER JOIN map ON images.tile_id = map.tile_id WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?", [11,387,769], function (res)
{alert('success');}, function () {alert('fail');});
It never returns anything, so it seems that the it never fires correctly.
Thanks for all the help,
Shane
Scott,
ReplyDeleteI used the build.phonegap.com interface to build the project, I am fairly new Mac, so this seem the easiest way. This site requires a config.xml, so this is what I used.
MB Tiles Tester
MB Tile Tester
I managed to get this running, building locally using Xcode.
ReplyDeleteHowever, it is running unusably slowly on my iPad 3 (even worse on an old iPad 1). When panning or zooming it takes several seconds to load new tiles, sometimes with the UI completely locked while it happens.
Is this the drop in performance you mentioned or have I managed to break something else ?
It was slow but not unusable for me on an iPad 2 as of this version of the app. Since then I've optimized the base64 encoding and it's very quick now. Even on an old 3GS. Unfortunately I cannot share that code. But you could probably figure it out. Dig into the SQLitePlugin and search for better ways to do the encoding.
DeleteI did get as far yesterday as deciding it was the base64 that was the issue. For the moment though for my current project I've just gone with putting the tiles on the filesystem, as I can get away with just shipping all the tiles I need in the app itself.
DeleteI am new to phonegap
ReplyDeletecan you tell me what is this line in your code
script src='http://localhost:8080/target/target-script-min.js#anonymous'
It's Weinre: http://people.apache.org/~pmuellr/weinre/docs/latest/
DeleteThank you for your quick response.
ReplyDeleteI am new to ios development, I try to do this example with phonegap 2.2.0 but its not work ( its loading like this http://twitpic.com/bhyabc ) can you please help me.
this is my xcode - http://twitpic.com/bhyat7
I haven't tried upgrading to 2.2.0 yet. I know that they changed some stuff with plugins. You may need to look at updating the sqllite plugin. If you get it figured out, please post your fix.
DeleteHmm.. I could not find any update to the sqllite plugin. Do you have any idea on where I can find a update for this? Can use Cordova framework sql lite instead?
ReplyDeleteYa, that plugin didn't seem like it was getting much love. The Cordova sqlite plugin may work as long as it is able to open specific files and can handle base64 encoded blob data. Please post your results if you have any success.
DeleteThanks!
Hi Scott,
ReplyDeleteI am trying to use your code in android (webview. i do not use phonegap or something like that). but it does not work.
i have two console.log. i see in log one row will be found. also database fetching is working.
if i check data length
console.log('data ' + result.rows.item(0).tile_data.length);
will be shown in log as 8.
but the tile images wont be shown.
what i am doing wrong? is something wrong with encoding
tile.src = base64Prefix + result.rows.item(0).tile_data;
thanks in advance.
L.TileLayer.MBTiles = L.TileLayer.extend({
//db: SQLitePlugin
mbTilesDB: null,
initialize: function(url, options, db) {
this.mbTilesDB = db;
L.Util.setOptions(this, options);
},
getTileUrl: function (tilePoint, zoom, tile) {
var z = this._getOffsetZoom(zoom);
var x = tilePoint.x;
var y = tilePoint.y;
var base64Prefix = 'data:image/png;base64,';
this.mbTilesDB.transaction(function(tx) {
tx.executeSql("SELECT tile_data FROM images INNER JOIN map ON images.tile_id = map.tile_id WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?", [z, x, y],function(tx,result){
console.log('Result****>' + result.rows.length + '<------' +'x; ' + x + ',y;' + y + ',z;'+z+'////////');
console.log('data ' + result.rows.item(0).tile_data.length);
tile.src = base64Prefix + result.rows.item(0).tile_data; //.rows.item(0).tile_data;
}, function (er) {
console.log('error with executeSql', er);
});
});
},
_loadTile: function (tile, tilePoint, zoom) {
tile._layer = this;
tile.onload = this._tileOnLoad;
tile.onerror = this._tileOnError;
this.getTileUrl(tilePoint, zoom, tile);
}
});
Have you tried logging the actual base64 data to make sure that you're getting it? Also, are you initializing the leaflet map when the page is visible? This can sometimes lead to blank maps.
Deletedid you use the SQLitePlugin https://github.com/chbrody/Cordova-SQLitePlugin in your android solution ? I didn't found how to map the .mbitles to this plugin ?
ReplyDeleteYes, I'm not sure if we used that fork of it or not. But that looks like the same one.
DeleteScott -- I have to thank you for getting me started here, it would have been more of an uphill battle to get where I am without being able to stand on your work. First I fixed up your example and the latest SQLitePlugin to work with the latest Phonegap release but as you noted, it was a bit slow... Considering it from an architectural point of view, it could possibly be sped up a bit by handing the queries off to threads if you can finaggle things to happen in parallel, or various other ways, but really it will always be slow, since you are forced to base64 encode the image data for every request. I looked for a better way, and not knowing the Android platform at all when I started it took a bit more time than it should have, but I found that webview's (what phonegap embeds) can access content:// url's exposed by a content provider written in Java and those content providers can return a descriptor representing a file to the webview rather than encoding it and doing a bunch of memory copies passing the data back as a string. I got a basic version of this implemented and working and it is very much faster indeed, and I think simpler too. Here's a link to the source, hopefully it helps some other folks out: https://gist.github.com/thesjg/4694437 -- would love to know if some analog of this is possible on iOS.
ReplyDeleteGreat work Scott and Samuel.
ReplyDeleteI'm doing an app with phonegap and have an issue of very bad 3g connection. It is for a map in the hills and I'm pretty sure I will have connection problems.
So my question is: how do I make everything locally? I mean, how do I get the map and import it via leaflet. So everyone who will own an app will download the app with map within, when there is still good connection and will have an ability to view maps offline later on.
Help would be much appreciated!
Samuel,
ReplyDeleteYour post sounds very promising. Please don't hesitate to share the project source. It would be very helpful for a newbie like myself. I feel that I'm close to understanding this process, but having trouble implementing the various aspects with my limited java coding skill. I hate to beg you :) but I'm just a lost wordpress developer hoping to make an offline map app for our geology dep.
All the best,
Ryan
Hi Scott - thanks for sharing this post. I'm using PhoneGap Build, so I couldn't use the SQLite plugin like you did. I did, however, manage to get it working with the raw tiles downloaded to the filesystem. Thanks for the inspiration!
ReplyDeleteBlog post: http://tech-blog.silviaterra.com/2013/02/offline-mapping-in-html5-mobile-apps.html
GitHub repo: https://github.com/SilviaTerra/offline_map_poc
Hello Scott,
ReplyDeleteI've been trying PhoneGap for the first time in the last days and I tried to reproduce your example. Like Gülsün, I'm stucked on tiles now showing the actual images. The requests work. Here's what the executeSql callback console log look like :
executeSql callback:[{"tile_data":"[B@40df1408"}]
I don't know much about blob formats and I've been searching around for ways to make the browser understand that, but I'm pretty sure that this is not base64 encoded. If not, how did you manage to make it work ?
I just read about Samuel's way of doing the same thing. This really looks like an interesting approach and I might try it as well, but I still wonder how you did it.
Ya, that doesn't look right. Do you have your SQLite plugin base64 encoding? This code may be helpful: https://github.com/stdavis/OfflineMbTiles/blob/master/OfflineTilesProof/SQLitePlugin.m#L181
DeleteHey Scott,
DeleteYep, that's exactly what was missing.
For the record, I also tried the content provider as suggested by Samuel and it also worked.
Thanks a lot to you all for sharing all this and for your time. That's much appreciated.
I have tryed your great example with cordova 2.2.4, but I get the next error, somebody solve it?
ReplyDeleteandroid.database.sqlite.SQLiteException: unknown error: Unable to convert BLOB to string
Sounds like you are missing the base64 encoding.
DeleteWow great example, really useful for me. Many thanks.
ReplyDeleteHey there, first, thanks for you work Scott, helps me out figure out how to do the same thing for Android.
ReplyDeleteI've updated the code to use the mapbox.js library which is based on Leaflet and upgrade to cordova 2.7, maybe it will helps some of you: https://github.com/tdurand/offline-phonegap-map
However i'm facing performance issue, when you zoom or move the map, it takes 1/2 seconds to render the new tiles, it really doesn't feel smooth....
As samuel did, i've tried to optimise the base64 encoding with faster library but it is still slow, Scott have you been able to have good performance with this base64 tile rendering method on Android ??
I'll try to implement samuel approach to see how are the performance.
Anyway, thanks a lot, i'll try to share my progress.
Yes, our performance in Android has been good. Unfortunately I'm unable to share our code. Maybe it's because we are using smaller base map databases (.mbtiles)? They are usually 20-100MB.
DeleteOkay thanks for your comment, don't worry, i've read that it was a close source project, i just wanted to know if it was possible to have good performance before diving more into that.
DeleteI'm working with your small mbtile file, i'll try with a larger one to see if it change something. I'll post further progress.
Thanks for this. Just went through getting it up and running in Cordova 2.8.1 with what seems to be the latest version of the plugin:
ReplyDeletehttps://github.com/j3k0/PhoneGap-SQLitePlugin-iOS
May write something of my own up on the process and some of the issues I ran into.
Please send me the link and I will post it.
DeleteI have implemented this POC on Android, as have others, but I am wrestling with a particularly strange problem - Leaflet seems to be calculating an incorrect Y coordinate (tile_row from SQLite DB file). I have tried this both with the SQLite DB file from your code as well as a few I generated from TileMill. I have tried setting various options in both the Map and the TileLayer.MBTiles objects, including making sure tms was set to true, setting maxBounds, center, zoom, minZoom, maxZoom, etc. I'm using version 0.6 of Leaflet. Wondering if you have any ideas?
ReplyDeleteI haven't seen that before. Although we are still at version 0.5 of Leaflet. Maybe there's something in the new version that's causing problems?
DeleteYes - that's exactly what the problem was (well, I had some other stupid mistakes as well!). At any rate, I finally have the demo running, and since it seems it would be helpful for other's wanting to implement the Android version, I have posted this to GitHub: https://github.com/kaidad/offline_map_demo
DeleteAnd just to clarify - the specific issue I found was that in Leaflet 0.6, the _loadTile and getTileUrl methods were modified. In _loadTile, _adjustTilePoint(tilePoint) was added and this was the missing piece with my code.
Delete@dave, i've tried your version but it is still painfully slow on my device ~2.5s to render when zooming or moving around.
Delete@scott it've tried with a larger mbtiles ~80mb, still really slow
Still investigating, will post further progress.
I've just got this running locally and offline using node.js and SQLite3 to return the pngs. The had a order problem that you solves here by setting TMS = true on the leaflet layer:
ReplyDeleteL.tileLayer('http://localhost:8080/tile/{z}/{x}/{y}.png',{
tms: true
}).addTo(map);
Also - be careful you pass through the right order Z - X - Y (zoom, row, columns)
Finaally, Scott, great write up but did you know the mbtiles sqlite file contains a view which simplies your SQL eg:
"SELECT tile_data FROM tiles WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?"
It's a minor detail but may help performance.
Cheers
Maybe it is a little bit too complex to use mbtiles package. Why don't you use MOBAC and generate your tiles in the OSMDroid format. Then you simply have to provide the tile structure in your web directory and to configure leaflet with template like 'maptiles/{z}/{x}/{y}.png'.
ReplyDeleteNo need for SQLite, no need for native plugin, simply pure HTML / JS.
Interesting. Not so great for downloading basemaps over the internet. But maybe if they were bundled with the app?
DeleteI have tried with Cordova as Christophe suggested, but it seems leaflet is not able to load the tiles from the web directory. Did anyone got this to work with Cordova?
Deletescott>>> can you develop for Android .. under contract. We are developing android based Navigation system. wangonduf@gmail..com
ReplyDeleteYes, just sent you an email.
Deletehello scott. i've built an android app for a town about 6 months ago. but offline map section has a native android code and causing some problems. i need phonegap one. can you help me?
DeleteI'm not an expert with Java and am booked for the next six months. All the best in your efforts.
DeleteHello Scott,
ReplyDeleteI have to develop an application in html5 that can serve maps in offline mode. My question is: Is it mandatory to use phonegap? Could I develop an application in html5 that synchronizes the tiles when online and then serves the maps locally?
Yes, I believe that you could do something similar with IndexedDB. However there are size restrictions on that database that you would have to check.
DeleteHi Scott
ReplyDeleteI have develop offline map App in phonegap.
I tried to modify your core.js (below). I want a local Database.db file
since I got already a lot of them :-) -- without downloading.
function buildMap() {
var db = window.sqlitePlugin.openDatabase({
name : "Database"
});
var map = new L.Map('map', {
center : new L.LatLng(41.311000, -72.927000),
zoom : 14
});
var lyr = new L.TileLayer.MBTiles('', {
maxZoom : 17,
scheme : 'tms'
}, db);
map.addLayer(lyr);
}
I got error.
[Switching to process 392 thread 0x2303]
[Switching to process 392 thread 0x207]
I am using this as a reference for in my phonegap android application. The probelm is that leaflet is loaded with blank image.
ReplyDeleteCan you gess the reason. Please help.
Hello guys, its a great approach, but in my case I'm working with openlayers(http://openlayers.org/) and I have to show aerial images that are stored inside the user's device. I've been searching and I can't find a solution yet because the openlayers just works if we pass a url like this "https://localhost:8080/{z}/{x}/{y}.png" but considering we're working with mobile device It don't have your own web server so I can't access the tile images just passing a localhost url right?
ReplyDeleteDo you know any approach in this case? Is it right create a web server on client device to take this tile images, my problem here is just to take the tiles from device.
Thanks in advance.
I'm not sure. What about a file-based url? Do those work for open layers?
DeleteActually I didin't try this way but I think that browser will not be allowed to take files from user's device, it will? or in this case FILE API solve this problem?
DeleteThanks for your attention
It works, I got these images in sdcard using file protocol. The issue I ran across was that when I tried to get the images using file protocol the browser showed cors error message, but it was just a seetting of properties in openlayers plugin.
DeleteThank you very much.