Furthermore....

On, and one more thing... The reason I felt I should give the lotto checker an update is because I've now got almost 25000 users.

Christmas Update!

It's been a while since I've last published something on here, but just thought I'd give a quick update. Been working on a couple of different things over the last while. I started working on a spring web app for configuring and training neural networks and I've also decided to do a re-write of the lotto checker app to give it a performance update and a functionality update.

The neural network project came from the fact that I have been reading a great book on neural networks in Java, as a refresher from when I last did AI and machine learning in college as part of an AI module and my final year project. I've always thought this area of computer science was one of the more interesting ones, and it by using spring it gives me a chance to hone my skills in spring development outside of work. Development is not planned and I usually only end up getting around to it if I'm too broke to go out on weekends, or if I'm bored because there's nothing on TV. Both of these situations are common enough for a good chunk of work to have been started on it already.

The lotto checker app has a number of games in it that are not being run any more so I've decided to give it a re-write. I have completely changed the design so that there's a process which is responsible for parsing the web page using regular expressions for each game type. This process puts the numbers in to a value object which a reusable screen component will be able to use for displaying the numbers. I've also decided to abandon any further development on the driver tools app as there are far too many other apps like it out there which are just as good (but with out the graphing). The code that I had written is not all lost as a lot of the GUI stuff in that app I will salvage for the lotto checker re-write.

I'm hoping by posting once more on a semi regular basis to this blog it will keep me motivated to do a bit more work on these projects. In the mean time Happy Christmas!

10,000 lotto checker downloads!

Today at some point my lotto checker app will have surpassed 10,000 downloads and to mark this event I'm working on an update. There's a number of things that I'm going to update, the first of which is that I'm now going to include ads in the app. I'm going to keep these small and they're going to only be on the results display page.

The second update is the ability to search for historic results. I'm changing this to appear with in a tabbed view in the app and it will work by posting data to the lottos website and parsing the HTML returned, or possibly displaying this with in some kind of browser view

For now I'm battling the API version problem with the latest google adSense jar. It seems it requires the SDK version 13 of android before it will display test ads.

QA machine

I've started the QA AI agent project and most of the bones and infrastructure for the application has been implemented at this point. The high level functionality of the bot is as follows.

High level QA bot flow chart

The bot should greet a user with a random greeting and wait for input. It takes the input and tries to extrapolate some kind of meaning from that input so it can perform a look up for any material that might be relevant. The bot should then create a response to the user using the most likely relevant material to the original input and send it back to the user. At each step the bot should look for any kind of exit conditions so it can output a sign off message.

So far the bot greets users and can identify a number of exit points, such as the user saying "goodbye" or "bye" or something of that nature. The responses are randomly picked from a XML file containing all of the conversation data. The XML looks as follows:

Conversation XML

Loading the conversation resources from an XML file is a design feature that I've added so that the addition of new lines that the box will use can be added with out having to rebuild the project. It might be possible then to include attributes to the elements to add semantic data for when those lines should be used. This could used to give the bot moods or even for localisation. The XML will change I'm sure in the future to enrich the data for each entry, but for now the structure is satisfactory. The conversation tag is the parent tag for conversation data, and each child group relates to some part of conversation such as a greeting, a goodbye, a follow up and so on. The data set that the bot looks for relevant information from could take the answers from this file also but that's something I've to work on.

The only missing thing at the minute is the meaning extraction from the user input. This is quite obviously the part where all of the work is done, but I've yet to look in to what natural language processing framework tools might work best for this. Having read a couple of white papers on this so far I've a better idea of how to approach this, which is where the above came from, however in terms of the actual work that has to be done to tag meaning or to classify an answer based on the input I've still to work out the details and do a lot more research. For now most of the infrastructure is in place, and building simple chat functionality in to the bot is kind of fun too.

Upcoming projects

I've decided to start a couple of new projects that I'm planning on working on in my spare time. I'm not going to post up a schedule or anything because they're very much in the planning stages and they're going to be quite large projects.

The first project is a natural language processing agent that responds to questions with logical correct answers from a given dataset. There are a number of these across the web that respond to quite a large variety of general questions that are there to purely try and emulate humans to pass a Turing test. The focus of this project will be a more directed search for an answer given a key. The key in this case is parsed or extrapolated from the question. This project is more concerned with the classification of a correct answer or response given these keys, rather than the ability to extract information from a question and dynamically generate an answer in an attempt to fool someone who submits a question. This project will involve the use of machine learning techniques and natural language processing. I may need to look at what might be out there already to speed up development when it comes to parsing or extrapolating meaning from questions.

The second project I'm working on is an e-commerce project. I've been working on it now for about a week or two at this point. The project is a platform for integrating with a payment gateway that can be used for storing payer data and scheduling a recurring payment over a period of time. I'm trying to design this as a platform so that additional functionality can be built on top of it and any website can utilise it's features through some sort of an interface or API.

I'll picking a project to start with over the weekend and maybe do a bit of research in to some of the Java natural language processing tools available. I've used the NLTK in python before which was excellent and easy to use, however I'd like to stick with Java this time as I'm not great with Python. Update coming soon...

Perl, CVS and more!

Ever since I've learned perl I've been finding more and more uses for everyday automated tasks. It's actually a ridiculously useful language and I'm incredibly happy that I've learned it. Yesterday I wrote a program that updates a php script that with the latest external IP address of my home server. It also updates the time that the IP address was updated and then some logic is performed server side to determine whether or not the server is online at the time of a page request.

This means that when ever I would like to access my server at home remotely, due to the fact that my ISP assigns an IP address dynamically, I can always have the latest IP (within 15 minutes) to connect to and I will always know if the server is on or not. This is not only handy for having access to a home computer at all times but it means I always have access to my cvs server remotely as well.

I set up a CVS server partly because I wanted to test out the package manager on fedora 16, partly because I wanted a back up of all of the projects in my eclipse workspace on my laptop and mostly because I wanted to become more familiar with using CVS. I've now some hands on experience not only updating my CVS server with a bunch of my old projects, but I plan to get some experience out of this managing the CVS server from an administrative perspective

Oh, and Fedora is awesome too!

Perl

I started reading reading a great book on Perl this morning in an effort to teach myself Perl to be more productive at work. Perl seems like a great tool that could be useful for automating a lot things that I do on a regular basis. I write a lot of Java stuff for parsing text but it seems like Perl is definitely a better tool for that job.

I'm about half way through the book "Beginning Perl". Having stacks and lists built in to it's core and focusing on what operators are used to use relevant data types based on it's context is great. A day's reading and I think I've got enough to understand a lot of what I'm going to need perl for. I'll be moving on to sub-routines, object orientated programming with Perl, regular expressions and then I'd like to cover some system programming with perl. Standard I/O and network programming will also be useful.

Going to be doing a lot more this evening after the France and Scotland match.

Adding more functionality

I've spent this afternoon getting back in to developing my Driver Tool app having not contributed to this project in quite a while. I would like to get back working on this on a regular basis and try get it published on the android market place so that I can start using it.

Today I completed the functionality of the settings menu, for selecting either metric or imperial as a measurement system. A user will be prompted with this menu upon first installing the app. Once the user has selected their preference they are not prompted upon app start up again. They can access the options menu through the menu button. I've also included the functionality of adding and deleting a car with an options menu which operates more or less in the same way as the transaction adding and deleting functionality. I've give the user the option to enter in the manufacturer and car model themselves when adding a car. I may update this in the future to populate a selection of drop down menus from a database of makes and models. Due to the tedious nature of populating such a database I've left that out for now.

There are a number of things that need to be done still which I'm going to start on. The first of these things is the tyre pressure calculator. I'm going to get back in to OpenGL for this. I'll have to come up with a spec for this part of the app, because I've only a couple of concept ideas in my head right now. I suppose I should probably remove the tab and finish up testing the already implemented functionality to push for a release. One of the other things that still needs doing is to build non-place holder assets for the menus and tabs, and work on the styling of the app. I think I would like to change the background from the standard black.

Update

I haven't had much time to get back coding in a while unfortunately. Life is a bit hectic at the minute, and I always seem to be busy with something else. Hopefully things will calm down soon enough and I'll have more time to spend relaxing and writing a bit of code. I type this now as I fight the urge to fall asleep!

Live Stats

Google Analytics data

As I mentioned earlier, here's the latest google analytics data over the period that I've been logging.

Popularity by page.

The above image shows a figure for the total number of hits tracked for each page or screen in the app that a user can look at. Although there are only a number of activities or screens that are re-used in the app implementation, when ever the data for a different game type it is tracked as if it were a separate page on a website.

Mmmmmm... pie.

The above chart shows the game results look up relative to each other game type. Lotto is a clear dominator.

Stats over time per page.

These graphs plot the number of page tracks over time which is interesting, as a clear trend of when people actually use the app can be seen. If you note that the graph spike on Wednesdays and Saturdays for the Irish lotto. EuroMillions turns up on Fridays, however interestingly people checked Euro Millions results more on Saturday. The number of people visiting each page actually reflects pretty I think on how many people play each game from my experience selling lotto in my previous retail job.

Live stats

The last image is the live tracking section of google analytics. This is the most I've witnessed people using my app at a single point in time, although it's entirely possible that there have been more people using it at once and I just hadn't seen it. It's interesting watching as individual clicks on pages scroll across the screen in real time. Obviously it's better to see in video format than a still image, as the interface is animated.

Measurement systems

Over the last couple of days I've made a number of changes to the Driver Tools app. A lot of them were purely aesthetic, however the biggest change is the addition of a static settings class for storing information that is saved across sessions about the user's preference for either the metric or imperial system. As a result you can now view the stats information in either Miles and Miles per imperial gallon, or you can view it as kilometres and litres/100k.

The next thing that I need to do is include a menu for changing this setting and finish of the mechanic for only ever allowing a configuration dialogue to appear once upon first starting up the app. Other areas that can be improved would be to graph or visualise more information on the stats page, and to clean up the fuel transaction log page. I also need to clean up the car picker menu and possibly allow a user to add a picture to their car.

Once those changes have been made and I finalise all of the pages or data that I'm going to track with analytics the app should be ready for release on the Android market.

Speaking of google analytics, I've built up a full weeks worth of stats and they're interesting. I'll write a full post on this at a later point with pretty pictures and graphs and stuff.