Recently I've been researching a mobile project that will require offline-capable base maps. After unsuccessfully finding a solution already built I decided to try wire together Phonegap, Leaflet, and TileMill's .mbtiles. After a few late nights I was able to see it come together and sent out a quick tweet. This has generated quite a few requests for more information so I threw up a repo on GitHub to demo what I was able put together.
The main idea is to download the .mbtiles to the device using Phonegap's File API. Since .mbtiles are just sqlite databases I was able to use a SQLitePlugin to open them up (thanks, coomsie!). I did run into a problem getting this plugin to read BLOB fields from the .mbtiles database. However, after a bit of poking I was able to get it to work.
Once I had access to the encoded image data it was only a matter of writing a custom leaflet tile layer (TileLayer.MBTiles.js) inspired by a similar one that coomsie had done. One of the big secrets was passing
{scheme: 'tms'} into the constructor.Initially the performance was quite good on the iPad 2 and the iPhone 4 that I tried it out on. However, after cleaning up the project in preparation for uploading it to Github, the performance has suffered a bit. Not sure what I did, but I'll post an update when I get it figured out.
This is my first experience with Phonegap and Objective-C so any suggestions for improvements would be greatly appreciated.
[Added on 4-26-12]
P.S. The app downloads the .mbtiles file automatically from my dropbox account the first time that it runs and stores it in the Documents folder locally. Each subsequent time that it runs it uses this local file. So after the initial download you should be able to open up the app and see tiles while in air-plane mode.
Cool, I've been wanting to do the same thing! Don't hesitate to put up more info.
ReplyDeleteIt would be cool to get this to run on android as well, but I guess one would need another sqlite plugin (plus, leaflet and android don't like each other that much at this point, it seems).
Yes, getting it to work on Android is my next step. I plan to check out this for a phonegap sqlite plugin: https://github.com/chbrody/Cordova-SQLitePlugin
DeleteHi Scott,
DeleteDo you still plan to get this work on Android? I've used chbrody's Cordova-SQLitePlugin (instead of coomsie's but I had to add some code from coomsie's) to have your code working and running on my iPad with Cordova v1.8.1, but haven't tested it on Android. I'm hoping it doesn't require too much changes.
Could you please let me know if you have made it work on Android or if you have any suggestions? Thanks so much in advance.
Yvette,
DeleteWe did get it working in Android. However, it was done as part of a project under contract so we can't share the code. The iOS version that I reference in this post was something that I did on my own before we had a contract signed. I did this so that I could share it. :)
Thanks Scott!
DeleteI appreciate the information. I will try to figure it out on my own. :)
Hi Scott...thanks for the post. Really helped me out and works well (on Cordova-1.9.0.js). I've got one problem though!
ReplyDeleteWhen extracting the tile images from the local sqlite db (in the L.TileLayer extension), I get the right tile data as a JSON object but cannot seem to turn the tile_data blob back into a real image (this is based on Android using the Cordova-SQLitePlugin code). Seems to make an invalid image file. I've not tried the iOS version (yet!).
Could this be related to the Android SQLitePlugin code for reading the blob data out of the DB (similar to the issue you had a in #3)? Any thoughts?
Laine
Actually scratch my last question - fixed now (I wasn't decoding the blobs coming back from the DB correctly, now encoding them to base64 strings and all works correctly).
ReplyDeleteGlad to hear that you got it figured out.
DeleteHi Laine,
DeleteCould you please post your fix here or post your project on Github? I'm going to try the code on Android and it would really help me out.
Thanks in advance!
Hello Scott... i tried your repo it is running fine but not showing the map just only the leaflet frame with a zoomin zoomout controle. what could be the possible problem can you help me
ReplyDeleteSounds like there is something wrong with your database. You might want to add some log statements to make sure that you are getting the base64 encoded data from the plugin successfully.
DeleteHello Scott how are you! Thanks for reply
ReplyDeleteI tried some alerts in plugin.js file but not every alert appears
this.openSuccess || (this.openSuccess = function() {
alert("DB opened: " + dbPath);
});
this.openError || (this.openError = function(e) {
alert(e.message);
});
these alerts never appears neither the alert inside SQLitePluginTransaction.prototype.complete.....
I also applied break points in objective c plugin files but only
-(CDVPlugin*) initWithWebView:(UIWebView*)theWebView
function is called out...
In Xcode console i am getting the following log
-[AppDelegate getCommandInstance:]: unrecognized selector sent to instance 0x913fba0
*** WebKit discarded an uncaught exception in the webView:decidePolicyForNavigationAction:request:frame:decisionListener: delegate: -[AppDelegate getCommandInstance:]: unrecognized selector sent to instance 0x913fba0
I am too dumb to solve this problem please suggest me a solution
Thanks for your help
Sounds like you need to go back in the process even further and confirm that the .mbtiles file was downloaded successfully.
DeleteI re-download the mbtiles files from your dropbox link but still it is just showing the leaflet frame, though i am not able to see the map but when i drag the map[invisible map :(] or zoom in or out its making the calls and i am getting the alerts.. may be these alerts are of loading the tiles cause number of times same alert appears is different....
ReplyDeleteThanks appreciate your support
Just in case.. I using cordova 1.9.0
DeleteNeev,
DeleteWere you able to iron this out, I am getting the same error, or issue.
Shane
I realized that the Plugin was not working right the way I was using the build.phonegap.com, looks like I need to test it via x-code.
DeleteNot really sure what's going on. I would try and log the output from the sqlite plugin and make sure that it's getting the encoded data. Then you could paste it into an online service that will translate it into an image to make sure that it's getting good images. Also, make sure that the map div is visible when you are creating it.
ReplyDeleteGotcha, It looks like the sqlPlugin ins the one that isn't working, if I do this:
ReplyDeletedb.executeSql("SELECT tile_data FROM images INNER JOIN map ON images.tile_id = map.tile_id WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?", [11,387,769], function (res)
{alert('success');}, function () {alert('fail');});
It never returns anything, so it seems that the it never fires correctly.
Thanks for all the help,
Shane
Scott,
ReplyDeleteI used the build.phonegap.com interface to build the project, I am fairly new Mac, so this seem the easiest way. This site requires a config.xml, so this is what I used.
MB Tiles Tester
MB Tile Tester
I managed to get this running, building locally using Xcode.
ReplyDeleteHowever, it is running unusably slowly on my iPad 3 (even worse on an old iPad 1). When panning or zooming it takes several seconds to load new tiles, sometimes with the UI completely locked while it happens.
Is this the drop in performance you mentioned or have I managed to break something else ?
It was slow but not unusable for me on an iPad 2 as of this version of the app. Since then I've optimized the base64 encoding and it's very quick now. Even on an old 3GS. Unfortunately I cannot share that code. But you could probably figure it out. Dig into the SQLitePlugin and search for better ways to do the encoding.
DeleteI did get as far yesterday as deciding it was the base64 that was the issue. For the moment though for my current project I've just gone with putting the tiles on the filesystem, as I can get away with just shipping all the tiles I need in the app itself.
DeleteI am new to phonegap
ReplyDeletecan you tell me what is this line in your code
script src='http://localhost:8080/target/target-script-min.js#anonymous'
It's Weinre: http://people.apache.org/~pmuellr/weinre/docs/latest/
DeleteThank you for your quick response.
ReplyDeleteI am new to ios development, I try to do this example with phonegap 2.2.0 but its not work ( its loading like this http://twitpic.com/bhyabc ) can you please help me.
this is my xcode - http://twitpic.com/bhyat7
I haven't tried upgrading to 2.2.0 yet. I know that they changed some stuff with plugins. You may need to look at updating the sqllite plugin. If you get it figured out, please post your fix.
DeleteHmm.. I could not find any update to the sqllite plugin. Do you have any idea on where I can find a update for this? Can use Cordova framework sql lite instead?
ReplyDeleteYa, that plugin didn't seem like it was getting much love. The Cordova sqlite plugin may work as long as it is able to open specific files and can handle base64 encoded blob data. Please post your results if you have any success.
DeleteThanks!
Hi Scott,
ReplyDeleteI am trying to use your code in android (webview. i do not use phonegap or something like that). but it does not work.
i have two console.log. i see in log one row will be found. also database fetching is working.
if i check data length
console.log('data ' + result.rows.item(0).tile_data.length);
will be shown in log as 8.
but the tile images wont be shown.
what i am doing wrong? is something wrong with encoding
tile.src = base64Prefix + result.rows.item(0).tile_data;
thanks in advance.
L.TileLayer.MBTiles = L.TileLayer.extend({
//db: SQLitePlugin
mbTilesDB: null,
initialize: function(url, options, db) {
this.mbTilesDB = db;
L.Util.setOptions(this, options);
},
getTileUrl: function (tilePoint, zoom, tile) {
var z = this._getOffsetZoom(zoom);
var x = tilePoint.x;
var y = tilePoint.y;
var base64Prefix = 'data:image/png;base64,';
this.mbTilesDB.transaction(function(tx) {
tx.executeSql("SELECT tile_data FROM images INNER JOIN map ON images.tile_id = map.tile_id WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?", [z, x, y],function(tx,result){
console.log('Result****>' + result.rows.length + '<------' +'x; ' + x + ',y;' + y + ',z;'+z+'////////');
console.log('data ' + result.rows.item(0).tile_data.length);
tile.src = base64Prefix + result.rows.item(0).tile_data; //.rows.item(0).tile_data;
}, function (er) {
console.log('error with executeSql', er);
});
});
},
_loadTile: function (tile, tilePoint, zoom) {
tile._layer = this;
tile.onload = this._tileOnLoad;
tile.onerror = this._tileOnError;
this.getTileUrl(tilePoint, zoom, tile);
}
});
Have you tried logging the actual base64 data to make sure that you're getting it? Also, are you initializing the leaflet map when the page is visible? This can sometimes lead to blank maps.
Deletedid you use the SQLitePlugin https://github.com/chbrody/Cordova-SQLitePlugin in your android solution ? I didn't found how to map the .mbitles to this plugin ?
ReplyDeleteYes, I'm not sure if we used that fork of it or not. But that looks like the same one.
DeleteScott -- I have to thank you for getting me started here, it would have been more of an uphill battle to get where I am without being able to stand on your work. First I fixed up your example and the latest SQLitePlugin to work with the latest Phonegap release but as you noted, it was a bit slow... Considering it from an architectural point of view, it could possibly be sped up a bit by handing the queries off to threads if you can finaggle things to happen in parallel, or various other ways, but really it will always be slow, since you are forced to base64 encode the image data for every request. I looked for a better way, and not knowing the Android platform at all when I started it took a bit more time than it should have, but I found that webview's (what phonegap embeds) can access content:// url's exposed by a content provider written in Java and those content providers can return a descriptor representing a file to the webview rather than encoding it and doing a bunch of memory copies passing the data back as a string. I got a basic version of this implemented and working and it is very much faster indeed, and I think simpler too. Here's a link to the source, hopefully it helps some other folks out: https://gist.github.com/thesjg/4694437 -- would love to know if some analog of this is possible on iOS.
ReplyDeleteGreat work Scott and Samuel.
ReplyDeleteI'm doing an app with phonegap and have an issue of very bad 3g connection. It is for a map in the hills and I'm pretty sure I will have connection problems.
So my question is: how do I make everything locally? I mean, how do I get the map and import it via leaflet. So everyone who will own an app will download the app with map within, when there is still good connection and will have an ability to view maps offline later on.
Help would be much appreciated!
Samuel,
ReplyDeleteYour post sounds very promising. Please don't hesitate to share the project source. It would be very helpful for a newbie like myself. I feel that I'm close to understanding this process, but having trouble implementing the various aspects with my limited java coding skill. I hate to beg you :) but I'm just a lost wordpress developer hoping to make an offline map app for our geology dep.
All the best,
Ryan
Hi Scott - thanks for sharing this post. I'm using PhoneGap Build, so I couldn't use the SQLite plugin like you did. I did, however, manage to get it working with the raw tiles downloaded to the filesystem. Thanks for the inspiration!
ReplyDeleteBlog post: http://tech-blog.silviaterra.com/2013/02/offline-mapping-in-html5-mobile-apps.html
GitHub repo: https://github.com/SilviaTerra/offline_map_poc
Hello Scott,
ReplyDeleteI've been trying PhoneGap for the first time in the last days and I tried to reproduce your example. Like Gülsün, I'm stucked on tiles now showing the actual images. The requests work. Here's what the executeSql callback console log look like :
executeSql callback:[{"tile_data":"[B@40df1408"}]
I don't know much about blob formats and I've been searching around for ways to make the browser understand that, but I'm pretty sure that this is not base64 encoded. If not, how did you manage to make it work ?
I just read about Samuel's way of doing the same thing. This really looks like an interesting approach and I might try it as well, but I still wonder how you did it.
Ya, that doesn't look right. Do you have your SQLite plugin base64 encoding? This code may be helpful: https://github.com/stdavis/OfflineMbTiles/blob/master/OfflineTilesProof/SQLitePlugin.m#L181
DeleteHey Scott,
DeleteYep, that's exactly what was missing.
For the record, I also tried the content provider as suggested by Samuel and it also worked.
Thanks a lot to you all for sharing all this and for your time. That's much appreciated.
I have tryed your great example with cordova 2.2.4, but I get the next error, somebody solve it?
ReplyDeleteandroid.database.sqlite.SQLiteException: unknown error: Unable to convert BLOB to string
Sounds like you are missing the base64 encoding.
DeleteWow great example, really useful for me. Many thanks.
ReplyDeleteHey there, first, thanks for you work Scott, helps me out figure out how to do the same thing for Android.
ReplyDeleteI've updated the code to use the mapbox.js library which is based on Leaflet and upgrade to cordova 2.7, maybe it will helps some of you: https://github.com/tdurand/offline-phonegap-map
However i'm facing performance issue, when you zoom or move the map, it takes 1/2 seconds to render the new tiles, it really doesn't feel smooth....
As samuel did, i've tried to optimise the base64 encoding with faster library but it is still slow, Scott have you been able to have good performance with this base64 tile rendering method on Android ??
I'll try to implement samuel approach to see how are the performance.
Anyway, thanks a lot, i'll try to share my progress.
Yes, our performance in Android has been good. Unfortunately I'm unable to share our code. Maybe it's because we are using smaller base map databases (.mbtiles)? They are usually 20-100MB.
Delete