Blog updates

Thursday, 12 January 2012

Chromemarks Updated - 2012.01.12

To continue with my new years' resolution of smaller, quicker updates - it's time for an update!
  1. Added a new Tutorial or "Quick Start" screen. This is only shown for brand new installs. It shows briefly how to perform the first sync, some common customisations and some common problems. Also uses the fancy swipey new ViewPager with a cute title indicator across the top. Also a nice offset image of the ChromeMarks logo behind the first screen - looks nice and arty.. Screenshots:

  2. Some fixes for the new OAuth2 process. The OAuth2 process relies on sending back the tokens in the title of the webpage which is easy for my app to pick up and then use. Unfortunately it looks likethere are a few devices out there that are returning *no* title. This makes the OAuth2 quite pointless on these devices! I've found another way to try it whereby the token is returned in the url instead and that's easy to grab. Hopefully that should work on all devices as a url is a url. Thanks to Curtis for being one of the first users with a Kindle Fire to kindly fill in his email address in the error box and happily test a fixed version for me.
  3. A mild tweak to the Google authenticator that was crashing if the user's token was more than 7 days old. Only had about 4 reports of this error, so it wasn't causing much trouble. Fixed nonetheless.
  4. Some tweaking to the size of the favicon shown for each bookmark and also when editing/creating the bookmark. I've made the icon one pixel smaller, but that now means it's rendered the exact size the icon is - this means the icon looks much crisper and cleaner. Hopefully the size decrease won't be too noticeable. Also tweaked the new "choose icon" popup to look better on higher dpi devices.
  5. Upgraded the Android Support/compatibility library to the latest version. I haven't spotted any issues arising from the previous one or the new so couldn't see any harm in doing this.
  6. Took the opportunity to revamp the help screens as well. Instead of having all the help links built into the app, the app now opens a webpage on an external website of mine. This allows me to easily change and reword the text and to add new answers should I get a flurry of new questions. This means we lose the translations for the original questions...but they still linked back to English answers anyway.. :-(
Next release is currently slated to be Honeycomb/ICS browser integration. Full app only. The ICS should be do-able, but can't guarantee Honeycomb as the source still isn't fully released. It'll be one-way at first, but some of the extra columns added by Google in the browser should help make this easier to make two-way at a later date.

I've had quite a lot of time off work recently which has helped with the recent releases - back to work again next week so releases may slow back down again.. :-/


Thursday, 5 January 2012

Chromemarks Updated - 2012.01.05

First new update of the year. Sorry it took so long but some of the changes had the potential to impact many users so I needed to take my time to test them more carefully.

As usual, the release notes:

  1. I've completely rewritten the network stack. There are two common ways of performing a network request in Android - apache http client and java net. Both have pros and cons and the app had been one for certain parts of logic and the other for other things. A while back a posting on the android developer blog suggested that they had been optimising one of these with each release of android and not touching the other one anymore.

    So I've made all network requests funnel through to a single place that determines the one method to use that is the most appropriate for your device (as suggested by Google on that blog). It's made the code a bit cleaner in many places as well. But just in case, I've added an override in the advanced settings that allows you to change the network stack to another one - use with caution as the app should pick the right one always and the different (i.e. not Google recommended) one may introduce other unknown errors.

    Preference to allow you to override the net work implementation. Use with caution!

    For users on Gingerbread and above, you may notice some performance benefits as the app is always using the more optimised version for your devices.
  2. Fixed a pet annoyance of mine. When creating/editing a bookmark in the full app, you could choose to fetch the icon. This did a round robin of three icon providers and picked the first that conformed to chrome's standards. Unfortunately they weren't always the prettiest icons returned by the providers - some times they were a bit washed out as the bpp had been lowered by the provider. There was also another provider out there that returned better quality icons, but wasn't in a format that Android could easily accept. I managed to find a way to accept the fourth icon provider, so now the app will allow you to see the results from all four. You can pick the one you like best and use that.

    Choose the icon to use. Click "view larger" to make them larger, but remember they are only 16 pixels so larger also means blurrier.

    May want a little more tweaking here as this all looks good on the emulator, but on my real phone the icons are a bit small and maybe hard to select - I think I'll need to add some scaling for device density to fix this. Also, they are requested one at a time and I'd like to try and ping off the request concurrently as if one provider takes a while to respond it holds them all up.
  3. For the full app, you can now add a widget to your home screen that will perform an immediate sync. It doesn't look pretty, but it works. My first foray into widgets - I still want to try and see if I can get a proper bookmarks widget sometime this year (but for users on 2.3 and below, there are no scrollable widgets so this adds a large amount of extra complexity). This "sync now" widget at the moment starts a sync itself. I may change that later to broadcast an intent which the main app responds to - this would then allow users to send that same intent from any other app they like, i.e. tasker, llama, etc.

    Rather boring screenshot of the new boring widget.. :-)

  4. Some hefty changes to the authenticator code. This is used to get tokens from Google to access your bookmarks and to authenticate/refresh them and authenticate to Chrome using those tokens. I've pulling it all out into it's own implementation. This allowed me to build an OAuth2 flavour as well. For users on non-google licensed devices (i.e. rockchip tablets, kindle fire, folio, etc), Google will not send back bookmarks to these devices if you authenticate via the android account. I'm guessing this is a large licensing block put in place. Over the last few months I've noticed some code going into Chrome to allow you to use OAuth2 to authenticate with Chrome instead. I don't know how to invoke it as the latest canary still uses the normal method of login - I'm guessing this might be there for chrome-os? Anyway, I've pinched some of the flags needed and now my app can do an OAuth2 request and retrieve your bookmarks. Now the standard Android account is still more secure, faster and most appropriate for accessing bookmarks, so please don't cut over to OAuth2 if the app already works for you. But for those who can't use the android account, this is the only decent alternative.

    NOTE: I still need some playtime - I'm not sure how this works with 2-step verification - I'm sure it does work, but I don't know which password to use yet. Hopefully this shouldn't break existing users authentication...:-)
  5. Some mild tweaks to the action bar for ICS users - before ICS the previous users with an action bar were wide tablets. Now with ICS, they might be mobile users and I noticed the action bar was getting mighty cramped and often the text wasn't all fitting in. I've tweaked this now to move the "sync" button to the main menu if there isn't enough space to display it normally.
  6. Another reason for the delays was that I wanted to try and get some proper testing suites up and running. Recently it's taken me longer and longer to test each release as I need to play on at least 5 different android emulators which might also further expand to include market and non-market devices depending on the changes. This takes a lot of time to keep repeating the same thing and if I make a mild tweak, I have to go back and run them all again and try to ensure I follow the same steps. So I've spent some time creating proper unit tests for some of the core functionality (simple sync tests, http tests, file export tests, etc, etc). It was a bit tricky to implement as I just couldn't get it to work in a seperate app like the documentation suggests (I had to keep adding more and more jar files to be exported and that just didn't seem right or automateable). In the end I've added them to the main app's codebase and my ant scripts that produce my builds simply hide or include that code. This also helped me fix a couple of bugs along the way in the authenticator and the main sync. Hopefully I'll add more over time.
  7. Some more tweaks to the backup functionality in the main app. Once again I was being stung by the android 1mb limit in a cursor, so I've split the data out and retrieve icons separately. Might mean a slightly slower export - especially if you have a lot of icons - but it should be more reliable. Also, some more fixes around the creation of the file itself whereby some errors to access the sdcard weren't being reported to the user properly.
  8. Also noticed at the start of December that the Lite app has been downloaded 250,000 times! Fantastic result! Also even better considering it only went on the market on November 28th 2010, so that's literally 250k in 12 months!
That's it for now. One new year's resolution is to *try* and release smaller, quicker. So the next release will either have some Honeycomb/ICS browser integration in it, or more likely a first-run tutorial with pretty pictures. It's likely to be the tutorial as it takes a lot of my time to respond to the emails from people who state "app is crap - no bookmarks found" and when I ask them "have you enabled sync in Chrome?", they then seem to go quiet... :-)

Monday, 2 January 2012

2011 / 2012

Thought it was time for an update on some of the key points over the last year with regards to ChromeMarks and what's on the horizon this new year.

2011's highlights.
  • In July, Google turned something off on the Chrome servers and the app started failing. 24 hours later, a fixed version was released. Honeycomb users are *still* waiting for their fix.. :-)
  • In Autumn, Chrome 14 arrived and with it's encryption, I had to frantically make updates to keep up with it as suddenly a load of people were being encrypted when they didn't even know about it (probably a bug in chrome enabled it for them).
  • Also during the autumn period, I was contacted by someone from Google who works on the syncing stuff. He let me know about impending changes. This was the first time anyone at Google has actually acknowledged the existence of this app and it was a very friendly exchange!
  • At the end of November, ICS was released with the fanfare that it can handle chrome bookmarks natively. However the number of purchases and users on ICS is steadily increasing. I've also had several mails from ICS users who can't get any bookmarks synced natively. Plus there are a couple of bugs raised on the android tracker from ICS users about this (hint: 23116, 22407)
  • In December, the lite app passed 250,000 downloads. It's been on the android market since 28th November '2010 - so in a year it has amassed this huge footprint.


The release of ICS gives me a new direction.

Since *some* users can now enjoy native bookmark syncing and many devices will be upgraded over the coming year to ICS, I need to change direction and start adding new functionality that ICS doesn't yet do. This has previously been stuff that I wanted to avoid - I hate those apps that bloat out with lots of functionality you don't need or care about (hint, hint, Nero Burning Rom, hint, hint). If I start adding more stuff to this app then I could start suffering with similar bloat. So I have to be careful in what I pick.

Below, I've listed - in no particular order - some of my hopes for the coming year. There's no guarantee they will all come to fruition, but I hope some will!

Honeycomb/ICS local syncing.

So Google released the source code for ICS near the end of the year. I was hoping to see the details of how they do the native sync (and see if I could supplementally fix some of it's issues and/or make mine better). However Google have implemented it within their closed source GoogleServicesFramework. This means the source is not available. And for ICS users with problems, this cannot be upgraded via the Market so requires a firmware upgrade to correct.

If I can get the app to work with Honeycomb/ICS stock browser then that allows the users with broken native syncing to have a fallback. Plus, I'm guessing that since Honeycomb/ICS has folder support built in, then Samsung will hopefully stop tarting about with Android and leave it as it is and I can finally support Samsung devices properly.

The number of 3.x and 4.x users is coming close to 10% of users for my full app. Ordinarily I wouldn't want to spend a large effort for something that 90% of users can't benefit from.

But I've also got a 3.x tablet, so I'd like to play and get this working for my own benefit as well... :-)

Password syncing.

There are some big hurdles to overcome here first.
  1. The Android Browser won't allow me to pass in a password. Or more succinctly, the browser will accept requests to start browsing and will read off the url only. Any other info I send with it is ignored. So the first hurdle is that even if I did have the username and password for a site, I can't "fire up" a new browser window and either autofill or autologin the user. Once the browser receives the request to navigate to a new web page, it's running under a different userid on the device and I can not communicate with it either.
  2. Since number one above screws things up a lot, another thought I had was to simply fire up the web address in a webview internal to my app. So Chromemarks would effectively become a web browser. Since it would be running in my own app, I now have control to autofill the fields. However don't underestimate the amount of work this would entail! There would be no tabs - unless I built them. There would be no long click menu - until I built it. There would be no zoom - until I built it. The list goes on. Suffice to say that the stock Android browser is a load of code wrapped around a webview - it's basically the same thing that I would need. The standard Eclair browser has 13,117 lines of code in it. The ICS browser has 39,400 lines of code. So users would expect at least the most basic of functionality, which would be a large amount of extra code. Just for auto-filling and using your passwords. That would take months just to badly copy it from Eclair - is that worth my time?
  3. For a while now in Android (I think from honeycomb onwards?), if you navigate to a webpage and enter an id and password, sometimes the browser will ask if you want it to remember it. So I wondered if I could simply sync your chrome passwords into that same repository so your browser automagically has remembered a whole load of new passwords. Unfortunately the database that holds these passwords is kept internal to the WebView - it is not exposed for *any* kind of data access from any other app. So I can't mass import/export.
  4. Another option I'm thinking about is to cheat and use a bookmarklet. An intriguing post I spotted on StackOverflow mentioned that a bookmarklet could have access to the content:// schema where normally a browser has tight security over what content can be used in a page. If this is true, a bookmarklet could dynamically retrieve a script from my content provider and load it into the current page. The content provider would be able to receive the incoming url, search the database, find the username and password and return them in a custom bit of javascript. Back in the browser, that bit of javascript is invoked which fills in the appropriate username and password fields. It sounds so simple - I'll bet it isn't! But if it works, this option would allow you to keep using your stock browser (and possibly any other browser that can run a bookmarklet?)
Real-time updates.

Chrome has a constant connection back to Google. It uses XMPP to push/pull notifications quickly. It's possible I could retro-fit this for my app, but I think it would be problematic at best. So I was thinking about writing some kind of chrome extension that could use C2D to send a signal to your device to tell it to perform a sync. This wouldn't be exactly real time, but within a few seconds, which is good enough! But I'm not sure if there is a way in Chrome to get notification that something has changed. Plus, a chrome extension is a whole new world! I'm proud that this app requires no third party extensions or software at all, so I'm a bit conflicted already..

Lite app.

Since ICS brings native bookmark syncing, I'm already bracing myself for the comments on the Lite app ("what's the point of this when my device can do this anyway - 1 star", "Google already does this - 1 star", etc, etc). Since I'm relying on the income from the full app, I'm thinking that the Lite app will probably continue as it is but without much more new functionality.

If I add new functionality in, then it will have to be stuff that ICS doesn't do to attract the users. And then I don't get the sales as the free app is doing it anyway. Sure I can move some of the advanced stuff to the full app, but it's starting to look tighter and tighter. And for those who whinge and say "just put ads in it", then you clearly aren't a normal developer. Yes if you built Angry birds then ads bring in a huge payback. But for normal developers, it's pathetic (hint: it wasn't even $100 for last month). Plus, the number of people out there who use Adfree or custom hosts files are increasing so the ad paybacks are always falling.

Suffice to say that the majority of the development for next year will be in new features and functionality for the full app. Yes, bug fixes and routine maint will go into the Lite app. But the number of automated bug reports has reduced to a very slow trickle now.

Chrome for Android

I keep an eye on all commits for Chrome so I can see if anything regarding the syncing is being updated. I see the old commit for Android usage. Although it still looks like a very long way to go before a decent Android build is available, I still have to remember that once that arrives, this app will likely die shortly afterwards. There won't be much more that I can do that a native Chrome won't already do - and for free.

But it's still a way off yet - I think they will probably have to go the same way as Fennec did recently: It tried to use the exact same codebase for the desktop version but the UI was slow, bloaty and intensely memory hungry. They changed to use native android widgets (instead of their xul), but it's a huge rewrite for a lot of code they never thought they would need to rewrite (as it's worked on all the previous operating systems Firefox has run on). I think Chrome will likely have to do similar - although I wouldn't put it past Google to implement some craftily helpful UI widgets in the next release of Android that Chrome could use very quickly and help get upto speed faster.

Everything else already on the list

  • Scheduler for the backup functionality. Backup before a sync, after each sync, multiple backups, etc, etc. Maybe even different formats - are there any other browsers out there that might become compatible if I provide an export for them?
  • First start tutorial. A user mailed me recently to say he had never pressed the "sync" button as he didn't know what it would do - he'd always used the auto-sync to keep his bookmarks updated. I really do need a first start tutorial with some screen shots to hint to new users what they can do! I've already sketched them up, just need the time.
  • Search engine integration with the Android Search widget.
  • A bookmarks widget. Regular request, but again quite a hard one to actually do. I've recently had a play with writing a widget, so I've got some ideas. I thought I could pinch the stock android bookmarks widget from ICS (the only version that currently handles folders) and use my database instead. However that relies heavily on new features from Honeycomb and ICS so won't work in Gingerbread which is easily the most used version. Plus, native scrollable widgets aren't possible unless you are using Honeycomb or a third party launcher, which further complicates matters.

That's it for now. Hope you have a happy new year!