OCR for Seven-segment display numbers in objective C.

The goal: Implement OCR for https://en.wikipedia.org/wiki/Seven-segment_display numbers in objective C.

The first step was to choose an OCR library. Tesseract https://github.com/gali8/Tesseract-OCR-iOS was an easy choice, being open source and with a lot of examples and resources available. For the training data I spent a lot of time playing with http://vietocr.sourceforge.net/training.html and https://code.google.com/p/tesseract-ocr/wiki/TrainingTesseract3 but at the end none of my training data worked better than this one https://github.com/arturaugusto/display_ocr. Anyways I recommend to try at least once the training process, as understand how Tesseract source training data works helps a lot with the image processing that will be required later.

The next step was to provide Tesseract with the source image. In my case it was a picture from the camera, so a bit of preprocessing was necessary, to allow the user to crop the text area. I used https://github.com/jberlana/JBCroppableView with the next modifications:

  • Force only 4 points
  • Recalculate the 4 points location on panning so they always keep a rectangle shape. There are two reasons for this, the more important one is that at the end, we are going to feed Tesseract with an UIImage, which is a rectangular context, and if the content is not rectangular as well we would need to colorise the not “image filled” background corners. Tesseract likes homogeneous backgrounds! The second reason is that for the user is much more easier to select the area as he needs only to drag two opposite corners of the selector to form the rectangle.

To make this the easiest way is add a tag to the points of the selector in the addPoints method of the JBCroppableLayer

so we can on the pan gesture in JBCroppableImageView and compare with the _pointsView.activePoint. Once we know which point we are moving is pretty straightforward to recalculate the position of the other three, alway carefully checking to not pass the image boundaries or user will not be able to pan back the points to the image. Example for the first point (top left).

I’m sure there is a more elegant compact way to do this, but in this case the main priority was a clear and easy to read code.

Once we have our cropped image it starts the more tricky part. Feed it as is to tesseract usually won’t work, and we will need to process the image to remove noise and highlight the text from the background. In my case the combination that worked better at the end was:

  • Convert to grayscale
  • Increase contrast in a factor of 7, so we end with an almost pure white and black image https://github.com/coryleach/UIImageAdjust
  • Expand image increasing the blank space between the borders and the text by at least 25%. This ended to be critical and was the ‘magic touch’ that made the OCR to work almost every time.
  • Erode the image to remove space between segments. Good hints of how to implement this in https://github.com/shu223/vImageCategory

If someone is interested in more details of this part, feel free to contact.

There were also a few Tesseract values that I played a bit with until I found the combination that worked better.

Custom views with dynamic content inside fragments (Android)

Today I have found an obscure problem that took me a few hours to figure it out and solve. I will write here how I have solved it, hope it can be of some help for some desperate developer.

Scenario

We are developing an Android app that a given moment has this structure:

Activity
—-Fragment
——–Sliding Tab Layout
—————-Fragment
————————Custom Views
——————————–Dynamically added views

Looks complicated, but is simple, think in a content fragment for a Navigation Drawer structure that contains a Sliding Tab Layout. The fragment inside one of the tabs of the Sliding Tab Layout contains a custom image gallery, created by extending a HorizontalScrollView. To that HorizontalScrollView we dynamically add ImageViews and TextViews to represent the images and their captions. Apart of the custom galery we are using other different customviews like a ViewPager and a TabView that follow a similar structure (in these cases the contents are directly defined in the XML layout)

The problem

When the fragment was loaded the first time, everything worked ok and was properly displayed, but when the main fragment was replaced and loaded again the views added dynamically inside the customviews didn’t appear.  The problem was somehow with the fragment internal loading, as the custom views worked ok when placed directly at the activity level.

Solution

After a few hours of trying different solutions that partially worked and involved nasty workarounds and hacks, the final (almost clean) solution was to delay the add of the custom views internal dynamic content using a Runnable

My guess is that the internal reference to the Layout generated during the initial fragment creation was somehow internally stored and not properly released during the fragment replace, and Android was trying to recreate it on the next attach before the view creation. The runnable makes the add to take place after the view creation and then the views are properly added to an existing parent view. Anyways as I said is a guess, but somehow related with the fragment problem.

Custom CRSSimple 1:1 proyection with leafleft.js

Some experiment with leafleft.js. Following the issue here is the core code for a 1px:1point working example for a finite map.

Important: All the (px,py) pixel coordinates  should be transformed  to standard LatLng coordinates using

Strangely, the

is basic to make the map work properly

Some thoughts after (almost) a year of real Xamarin use

Ten months ago at my team we reached that point that I think any mobile developer know about. Is it worth developing native? Should we keep coding the same things two/three times? Master a programming language and a technology is difficult, think about three. Not to mention to try to keep updated with the continuous incoming of frameworks, versions, libraries and components.

This is, of course, something inherent to software development in general, not only mobile. The thing is that even that we daily need to deal with different browsers nonsense in web development, or with multiple OS versions and environments in server/desktop, those are (usually) implementation details. We don’t usually need to re-implement the whole app and maintain separate codebases for Chrome and Firefox.

So, and finishing the self-pity for today, we tested most all the code-one-deploy-all available environments at the time (Cordova, Marmalade Phonegap, Appcelerator…) and we finally decided to give a serious try to Xamarin. The main reasons? We wanted the most native approach, not HTML5 web wrappers, we had lot of experience using C#, and the Mono-.NET framework ecosystem availability was too tempting. (Once you know linq is hard to renounce to it).

Now is time to decide to renew or not the Xamarin subscription and following the internal discussion we are having here, these are my conclusions from a programmer and software architect perspective.

PROS

  • Xamarin works. During this time we have developed, tested and deployed Android and iOS applications using most of the possible features a mobile app can  target, among others:
    • Google play and Apple store deployment
    • In-house deployment with custom autoupdates
    • Push notifications
    • Custom and remote web content loading with both ways native-javascript interaction
    • Beacons technology and real time signal processing
    • Any kind of imaginable UI layout
    • PCL development and background code sharing
    • Automated testing using Calabash
    • Video streaming
    • OAuth integration
    • Remote REST APIs consuming
    • Social media (facebook, twitter) integrations
    • Embebbed SQLite
    • XMPP
    • Phone and tablet deployment

    So, our general feeling is that if is possible to do it native is possible (I don’t say easy) to do it using Xamarin.

  • The Mono framework integration works. Yes, you really can use C#, linq, Lists, Generics and -most- of your beloved .net stack
  • Growing ecosystem. The components library keeps improving and getting more and more useful stuff.
  • Growing project  and community. I don’t know of course the internal situation of the company, but seems that the Xamarin project is alive and has future. There are continuous  developer meetings in cities all around the world and they keep pushing new improvements in the product.
  • Microsoft seems to start to seriously support the project.

CONS

  • Yes, you can make the same things that you make using java or objective c or swift, but when your app needs to go a bit further in terms of UI or interaction, things are not straightforward. We develop very specific and customized boutique apps with intense requirements in terms of design, animation and integrations and when you try to push that forward in Xamarin problems start to appear. (Missing features, compatibility issues, not existing libraries…)
  • Xamarin creates his own problems. As a new layer over the native environment you don’t have real control of what is generating as the final code that will be run on the device. There is the case of an iOS objective-c feature we translated to Xamarin, a very specific subclass customization of the UINavigation bar. In theory we were using only standard objective c instructions (like, add this UIView here, push that in the UINavigationController , change this color…) so the translated C# code was the same, line by line. But the Xamarin implementation didn’t work. We never got to know exactly what the problem was, but the order in what the UIViews were drawn  was important and we suspect that the code generated by the xamarin compiler wasn’t the same that the one generated using Xcode. We ended implementing a not pretty workaround.
  • Following this, xamarin has his own bugs. This is normal, of course, but the point is that you are in fact adding a new source of possible problems to your development process. We had a problem using the HttpClient class  that costed us a lot to track as a Mono implementation problem, and not ours. Again, we ended implementing a workaround against the standard use of HttpClient. Similar stories to this one regarding memory leaks,  not standard behavior, happened to us very often
  • All these required workarounds are something you cannot plan at the beginning of the projects, nor are problems you can expect if you use standard programming guidelines, and keep adding time, cost and effort.
  • Logically, the amount of resources, components, libraries and available help is far from the available for java or objective c. Situations like ‘I would use this cool slide menu library but the xamarin bind/port doesn’t exist, or is much older, or it would take a lot to implement’  or ‘this stack overflow thread will help me, wait is for native not for xamarin’ are common.
  • Android development is slow. Like painfully slow. Not the development itself, but the process of deploy to the emulator or to the real test device. It takes ages, no matter if you use the standard emulator or the Xamarin Android Player , or a real device.  No possible comparison with deploy using Android Studio, specially to real devices.
  • Price. This of course depends of your team size, resources and what you can expect to save using Xamarin, but 1000$ developer/year is not exactly cheap.
  • You can share background code, business logic… but again, if you want real customization you need to implement the UI yourself.  There is part of the business logic  that in fact you can copy, but no share, because the specifics of the flow execution in each platform. (As a very simplified sample, you cannot always make a 1:1 relation between Android activities and ios UIViewControllers and that ends with part of code that should be in different places)   Even with Mvvmcross (that we have used) is not always possible.
  • The solution to even share the UI (Xamarin forms) right now, doesn’t work apart of for simple apps and UI. Of course this is only our experience, maybe we are wrong, but if make something detailed or specific in UI using Monotouch or Mono for android is difficult, with Xamarin forms is simply not possible, and speaking about the code itself, you usually end with a bunch of separate interfaces with different code for each platform.
  • Xamarin studio, the development tool if you are using mac or dont have Visual Studio, its still far away of Xcode or Android Studio. It has improving during this time of course, and is usable  but it lacks the features of his native counterparts, and continuous crashes and mysterious file disappearances are routine, not to mention the fear of what will crash now after each system update.
  • The community (only part of it of course) sometimes takes the defence of Xamarin too serious, and you cannot go against the idea of how is the solution to all the mobile development problems . I mean, is a great tool I enjoy working with, but that’s it. If I found a better solution for a problem, I would use it

So, to sum up, is it Xamarin worth it?  For me yes, but as other tool to add to your options, and not to totally replace native development, at least for now. Depending of the project and requirements I would be happy to have Xamarin available and I will use it, but there will be other situations in which I really think use native will be a better choice.

A C# map tiles generator

To start, a little C# map tiles generator I have written for some custom maps (self contained coordinates system) we are developing using leafletjs . Nothing fancy, input image width and height should be multiples of 256px (no blank space management), but it solves a problem I found using gdal2tiles (intermediate tiles were generated in opposite positions!).

The parameter parsing was made using .NET CLI a C# port of Apache CLI