Furthermore....

On, and one more thing... The reason I felt I should give the lotto checker an update is because I've now got almost 25000 users.

Christmas Update!

It's been a while since I've last published something on here, but just thought I'd give a quick update. Been working on a couple of different things over the last while. I started working on a spring web app for configuring and training neural networks and I've also decided to do a re-write of the lotto checker app to give it a performance update and a functionality update.

The neural network project came from the fact that I have been reading a great book on neural networks in Java, as a refresher from when I last did AI and machine learning in college as part of an AI module and my final year project. I've always thought this area of computer science was one of the more interesting ones, and it by using spring it gives me a chance to hone my skills in spring development outside of work. Development is not planned and I usually only end up getting around to it if I'm too broke to go out on weekends, or if I'm bored because there's nothing on TV. Both of these situations are common enough for a good chunk of work to have been started on it already.

The lotto checker app has a number of games in it that are not being run any more so I've decided to give it a re-write. I have completely changed the design so that there's a process which is responsible for parsing the web page using regular expressions for each game type. This process puts the numbers in to a value object which a reusable screen component will be able to use for displaying the numbers. I've also decided to abandon any further development on the driver tools app as there are far too many other apps like it out there which are just as good (but with out the graphing). The code that I had written is not all lost as a lot of the GUI stuff in that app I will salvage for the lotto checker re-write.

I'm hoping by posting once more on a semi regular basis to this blog it will keep me motivated to do a bit more work on these projects. In the mean time Happy Christmas!

10,000 lotto checker downloads!

Today at some point my lotto checker app will have surpassed 10,000 downloads and to mark this event I'm working on an update. There's a number of things that I'm going to update, the first of which is that I'm now going to include ads in the app. I'm going to keep these small and they're going to only be on the results display page.

The second update is the ability to search for historic results. I'm changing this to appear with in a tabbed view in the app and it will work by posting data to the lottos website and parsing the HTML returned, or possibly displaying this with in some kind of browser view

For now I'm battling the API version problem with the latest google adSense jar. It seems it requires the SDK version 13 of android before it will display test ads.

QA machine

I've started the QA AI agent project and most of the bones and infrastructure for the application has been implemented at this point. The high level functionality of the bot is as follows.

High level QA bot flow chart

The bot should greet a user with a random greeting and wait for input. It takes the input and tries to extrapolate some kind of meaning from that input so it can perform a look up for any material that might be relevant. The bot should then create a response to the user using the most likely relevant material to the original input and send it back to the user. At each step the bot should look for any kind of exit conditions so it can output a sign off message.

So far the bot greets users and can identify a number of exit points, such as the user saying "goodbye" or "bye" or something of that nature. The responses are randomly picked from a XML file containing all of the conversation data. The XML looks as follows:

Conversation XML

Loading the conversation resources from an XML file is a design feature that I've added so that the addition of new lines that the box will use can be added with out having to rebuild the project. It might be possible then to include attributes to the elements to add semantic data for when those lines should be used. This could used to give the bot moods or even for localisation. The XML will change I'm sure in the future to enrich the data for each entry, but for now the structure is satisfactory. The conversation tag is the parent tag for conversation data, and each child group relates to some part of conversation such as a greeting, a goodbye, a follow up and so on. The data set that the bot looks for relevant information from could take the answers from this file also but that's something I've to work on.

The only missing thing at the minute is the meaning extraction from the user input. This is quite obviously the part where all of the work is done, but I've yet to look in to what natural language processing framework tools might work best for this. Having read a couple of white papers on this so far I've a better idea of how to approach this, which is where the above came from, however in terms of the actual work that has to be done to tag meaning or to classify an answer based on the input I've still to work out the details and do a lot more research. For now most of the infrastructure is in place, and building simple chat functionality in to the bot is kind of fun too.

Upcoming projects

I've decided to start a couple of new projects that I'm planning on working on in my spare time. I'm not going to post up a schedule or anything because they're very much in the planning stages and they're going to be quite large projects.

The first project is a natural language processing agent that responds to questions with logical correct answers from a given dataset. There are a number of these across the web that respond to quite a large variety of general questions that are there to purely try and emulate humans to pass a Turing test. The focus of this project will be a more directed search for an answer given a key. The key in this case is parsed or extrapolated from the question. This project is more concerned with the classification of a correct answer or response given these keys, rather than the ability to extract information from a question and dynamically generate an answer in an attempt to fool someone who submits a question. This project will involve the use of machine learning techniques and natural language processing. I may need to look at what might be out there already to speed up development when it comes to parsing or extrapolating meaning from questions.

The second project I'm working on is an e-commerce project. I've been working on it now for about a week or two at this point. The project is a platform for integrating with a payment gateway that can be used for storing payer data and scheduling a recurring payment over a period of time. I'm trying to design this as a platform so that additional functionality can be built on top of it and any website can utilise it's features through some sort of an interface or API.

I'll picking a project to start with over the weekend and maybe do a bit of research in to some of the Java natural language processing tools available. I've used the NLTK in python before which was excellent and easy to use, however I'd like to stick with Java this time as I'm not great with Python. Update coming soon...

Perl, CVS and more!

Ever since I've learned perl I've been finding more and more uses for everyday automated tasks. It's actually a ridiculously useful language and I'm incredibly happy that I've learned it. Yesterday I wrote a program that updates a php script that with the latest external IP address of my home server. It also updates the time that the IP address was updated and then some logic is performed server side to determine whether or not the server is online at the time of a page request.

This means that when ever I would like to access my server at home remotely, due to the fact that my ISP assigns an IP address dynamically, I can always have the latest IP (within 15 minutes) to connect to and I will always know if the server is on or not. This is not only handy for having access to a home computer at all times but it means I always have access to my cvs server remotely as well.

I set up a CVS server partly because I wanted to test out the package manager on fedora 16, partly because I wanted a back up of all of the projects in my eclipse workspace on my laptop and mostly because I wanted to become more familiar with using CVS. I've now some hands on experience not only updating my CVS server with a bunch of my old projects, but I plan to get some experience out of this managing the CVS server from an administrative perspective

Oh, and Fedora is awesome too!

Perl

I started reading reading a great book on Perl this morning in an effort to teach myself Perl to be more productive at work. Perl seems like a great tool that could be useful for automating a lot things that I do on a regular basis. I write a lot of Java stuff for parsing text but it seems like Perl is definitely a better tool for that job.

I'm about half way through the book "Beginning Perl". Having stacks and lists built in to it's core and focusing on what operators are used to use relevant data types based on it's context is great. A day's reading and I think I've got enough to understand a lot of what I'm going to need perl for. I'll be moving on to sub-routines, object orientated programming with Perl, regular expressions and then I'd like to cover some system programming with perl. Standard I/O and network programming will also be useful.

Going to be doing a lot more this evening after the France and Scotland match.

Adding more functionality

I've spent this afternoon getting back in to developing my Driver Tool app having not contributed to this project in quite a while. I would like to get back working on this on a regular basis and try get it published on the android market place so that I can start using it.

Today I completed the functionality of the settings menu, for selecting either metric or imperial as a measurement system. A user will be prompted with this menu upon first installing the app. Once the user has selected their preference they are not prompted upon app start up again. They can access the options menu through the menu button. I've also included the functionality of adding and deleting a car with an options menu which operates more or less in the same way as the transaction adding and deleting functionality. I've give the user the option to enter in the manufacturer and car model themselves when adding a car. I may update this in the future to populate a selection of drop down menus from a database of makes and models. Due to the tedious nature of populating such a database I've left that out for now.

There are a number of things that need to be done still which I'm going to start on. The first of these things is the tyre pressure calculator. I'm going to get back in to OpenGL for this. I'll have to come up with a spec for this part of the app, because I've only a couple of concept ideas in my head right now. I suppose I should probably remove the tab and finish up testing the already implemented functionality to push for a release. One of the other things that still needs doing is to build non-place holder assets for the menus and tabs, and work on the styling of the app. I think I would like to change the background from the standard black.

Update

I haven't had much time to get back coding in a while unfortunately. Life is a bit hectic at the minute, and I always seem to be busy with something else. Hopefully things will calm down soon enough and I'll have more time to spend relaxing and writing a bit of code. I type this now as I fight the urge to fall asleep!

Live Stats

Google Analytics data

As I mentioned earlier, here's the latest google analytics data over the period that I've been logging.

Popularity by page.

The above image shows a figure for the total number of hits tracked for each page or screen in the app that a user can look at. Although there are only a number of activities or screens that are re-used in the app implementation, when ever the data for a different game type it is tracked as if it were a separate page on a website.

Mmmmmm... pie.

The above chart shows the game results look up relative to each other game type. Lotto is a clear dominator.

Stats over time per page.

These graphs plot the number of page tracks over time which is interesting, as a clear trend of when people actually use the app can be seen. If you note that the graph spike on Wednesdays and Saturdays for the Irish lotto. EuroMillions turns up on Fridays, however interestingly people checked Euro Millions results more on Saturday. The number of people visiting each page actually reflects pretty I think on how many people play each game from my experience selling lotto in my previous retail job.

Live stats

The last image is the live tracking section of google analytics. This is the most I've witnessed people using my app at a single point in time, although it's entirely possible that there have been more people using it at once and I just hadn't seen it. It's interesting watching as individual clicks on pages scroll across the screen in real time. Obviously it's better to see in video format than a still image, as the interface is animated.

Measurement systems

Over the last couple of days I've made a number of changes to the Driver Tools app. A lot of them were purely aesthetic, however the biggest change is the addition of a static settings class for storing information that is saved across sessions about the user's preference for either the metric or imperial system. As a result you can now view the stats information in either Miles and Miles per imperial gallon, or you can view it as kilometres and litres/100k.

The next thing that I need to do is include a menu for changing this setting and finish of the mechanic for only ever allowing a configuration dialogue to appear once upon first starting up the app. Other areas that can be improved would be to graph or visualise more information on the stats page, and to clean up the fuel transaction log page. I also need to clean up the car picker menu and possibly allow a user to add a picture to their car.

Once those changes have been made and I finalise all of the pages or data that I'm going to track with analytics the app should be ready for release on the Android market.

Speaking of google analytics, I've built up a full weeks worth of stats and they're interesting. I'll write a full post on this at a later point with pretty pictures and graphs and stuff.

Lotto Checker stats, now with analytics results!

It's been an interesting few days with the inclusion of google analytics, however downloads is quickly approaching the 5000 marks, and with the inclusion of google analytics I now have a better idea of how people are using the app. The first day I updated my app to include analytics I saw that on Saturday 746 people started my app, with 452 on Sunday. I updated the app on Friday, so I would imagine that a lot of these hits were due to people checking out the app to see if there was anything new. I will have to observe the numbers over a longer sample period to gauge the weekly trends. It will be interesting to see next weekend and on Wednesday if the trend matches this Weekends usage, as the Irish lotto games are on Wednesdays and Saturdays.

I'm also looking at the pages that people are viewing. There were a total number of 1872 pages tracked on Saturday night, 778 of which were Lotto. An interesting observation is that my app has had more page view tracks than this blog has had in the number of months I've been posting to it. There were quire a large number of Lotto checks on Sunday as well at 498, probably because people might have resorted to checking numbers the day after the draw.

One of the most interesting parts of google analytics is the way in which you can present the data. The live data tracker is also quite impressive, where you can witness page events being tracked in real time on a sliding graph. It also gives you insight in to how many users were on the app at any given time.

I'm going to get back developing my Driver Tools app this week and try and get some debugging done on the graph information. There are a number of other design features I would like to review also, however I can't think of them at the minute. I'll have an update later on in the week.

Lotto Checker 1.5

I updated the lotto checker app today with the session class that I wrote for the Driver Tools app. It contained all the functionality for tracking the activities a user will start in my app. In the lotto checker app, I track when the app is started up, and when ever a user displays the results for a game, that is also tracked. This will help me in deciding on what an how I use analytics in the Driver tools app, and it will give me a better idea of how used my lotto checker app is, so that I can improve different aspects of it in the future.

The total number of downloads for my Lotto Checker app now stands at 4,373 total installs (users) and 3,028 net installs (devices).

Google analytics update and saving sessions. Back in the game!

These past two weeks that I've had off for Christmas have been the longest period of time that I've taken off work in the last 3 years. I didn't expect to enjoy doing literally nothing productive over the course of my time off, however it happened. I think I needed the break. I've had enough of doing nothing for now anyway, and I suppose my time away from development gave me a bit of time to think about the approach I was taking for this app.

I opened up a Claude Debussy playlist on iTunes and started coding again tonight and managed to get Google analytics working. I'm currently only tracking three pages, however once I decide on what exactly it is that I'm going to track, then I'll include more. Other changes include the addition of an abstract class for handling all of the session data. This is also used for tracking various pages. It's an abstract class with a number of variables which are encapsulated to make everything nice and tidy. The only other variable it stores for the minute about the session is the ID of the selected car. It also saves this to the app's settings upon creating and destroying the app.

I made a few slight changes to the database adapter, to include a function for getting all of the transactions for the currently selected car in the session. So that now it only shows all of the transactions and stats for the selectd car. I may include a feature to show the spending across all cars at a later point.

Also, happy new year!

Post Christmas update!

I haven't had much time to dive back in to the driver tools project yet, with all of the social events going on over the Christmas holidays. This is going to change by tomorrow. Either later on tonight or tomorrow morning, I'm going to figure out how to get google analytics working properly, and I'll be looking in to setting up some app data with regards to the car that a user will select. I'll have an update soon

Deadline status update.

Today is the deadline I set for a releasable version of this project. The app is divided in to four parts. It has a tool for logging fuel transactions, a tool for picking and displaying information about your car, a tool for displaying information graphically about a user's driving habits and a tyre section for optimising wheel size and tyre pressure for maximum performance and economy. So far I have a beta version of the transaction and graphing tools and as as of tonight I can say that the car picker tool is now out of alpha and in to beta. The tyre pressure calculator can be done at a later point, and I'm not all that concerned about that for the minute.

The app in it's current state could be released on the market, however the styling of the app is not currently at the level of that which was originally envisaged and as a result of this fact I will not release the app to the market yet.

Currently all of the core functionality for in the beta version of the app has been implemented, but before I release it I would like to include google ads and goolge analytics and then clean up the interface to make it look as good as it can. So to summarise, the app functionality requirements have mostly been met but I will not be releasing the app just yet.

Checklist update

I've added most of the functionality to the database adapter for pulling out and inserting the data to and from the database. There's still a bit of testing that needs to be done, and there are a number of tables that I just haven't bothered implementing yet. I'll update these at a later point.

I've also modified the car picker tab to now display all of the cars in a table format, much in the same way the fuel transactions are displayed. I may change this to a relative layout which would look a lot better. The basic functionality is there. I also still need to test this class fully.

I've also identified a number of short cuts I can take now, such as the car manufacturer and model tables in the database because I won't actually be using them yet. I've taken out the blob column in the car database until I can research if there's a better way of storing such images. But I can knock off two and a half items from my check list. More will be done tomorrow.

Structured development time and to do list

I saw this blog post on linkedin there and I can see how a number of the things mentioned in the article hinder productivity. In particular multi tasking. Multi-tasking is bullshit.

I'm going to list out a number of different things I'd like to get done tonight and tomorrow night. These will be small steps of implementation which will lead me towards completing the integration of the new database schema and hopefully it will get put me on time for release on Sunday night. Also, I'm going to try and code in blocks of an hour at a time, then go do something else for a bit, and come back. I always thought it was a good idea to go and take a break, which usually consisted of a cigarette break in college, and by the time I came back to my computer I would practically have the problem solved. When you look at something after taking a break, sometimes the solution just jumps out at you and it appears so obvious that you feel embarrassed you had a problem with it in the first place. I can definitely see structuring my time like this benefiting my development.

• Add method to database adapter for getting all cars as a list of cars, similar to the way transactions are retrieved.
• Add view for displaying cars in cars tab.
• Add method of adding, editing and deleting cars.
• Add session data so that app can store which car is selected. To be used for adding relating car to transactions.
• Fix transactions so that only those for the car that's selected for the session are displayed.

If I get half of these done tonight I'll be doing well. I'll try get a mix of the more complex and more simple ones done tonight, and do the same again tomorrow night.

Database changes and plans to save the state of the app.

It's been a while since I've touched anything databases related but I've made a number of changes to the database structure so that now it includes the following tables.

Brand

• Brand ID - primary key
• Brand Name

Model

• Model ID - primary key
• Brand ID - foreign key
• Model name

User Cars

• Car ID - primary key
• Model ID - foreign key
• Mileage
• Car Picture

I'm eventually going to add more columns to the Model table for some of the specifications for each model. I'll need to make a number of changes to the database adapter in order to store transactions with the new schema, so that when data is added to the database it adds it to conform with the new foreign keys.

In updating the database, the previous database was wiped and as a result I discovered that when viewing the stats tab when there was nothing in the database it would cause a null pointer exception. I amended the method for formatting the database results in to a Androidplot compatible data structure to only do so if there was data being returned.

The fact that the database wiped on an update to the schematics of the database got me thinking that I really need to look in to a way of converting the data that's currently in the database to form what ever the new schema is during an app update if the original schema is changed. There could well be an easy solution for all of this but as I stated above, it's been a while since I've got in to some solid database design.

Currently nothing is being done with these new tables, but I'm going to work on modifying the existing classes and their methods to fit the new schema this week, and I want to change up the car tab so that the main tab displays a list of the user's cars that he's added to their app (i.e. what will be stored in the UserCars table). The user can select which car the app uses which can be saved as part of the app's settings. This will link in to the transactions so that the mileage being displayed is for the selected car's mileage and the stats being displayed will have a relationship with individual cars in the user cars table too. I just need to change the user_id column in the transactions table to be a foreign key from the UserCars table.