ICFP 2012: Lambda Mining

This year's competition struck a familiar note -- elements on a grid, being updated every timestep. The premise of the problem was a worldwide shortage of "Lambdas" due to increased use of functional programming languages. The input was a map, consisting of earth, rocks, lambdas, etc. Your character (a mining robot) started in one location of the map, and the Lambda Lift (your target) started in another location. Your objective was to output a series of movements to collect as many Lambdas as possible, avoid rocks and other hazards, and ultimately arrive at the lambda lift for maximal points. For more details about the problem statement, see icfpcontest.org.

As a team of 5 MIT rising sophomores, we put together a solution revolving around a genetic algorithm. Below is the README file describing our algorithm which we attached with our solution:


Team IDKJava - 96121980
* Gurtej Kanwar
* Anthony Lu
* Will Oursler
* Gary Wang
* Tim Yang

Summary:
Looking at this kind of problem, a human would most likely break down the map into various smaller situations he knew how to solve, or which were easy to solve. Designing a program to follow these principles would have been much more difficult, especially as the new types of materials and effects were discovered throughout the competition. As such, we decided to go with a more general solution -- our idea was that by avoiding any casework at all, our code would be simpler and easier to extend as changes arose. By making use of randomness and many iterations in the form of a genetic algorithm, we were able to to do just that.

The genetic algorithm:
Our genetic algorithm was defined generally, to allow for easy variation. It operated on "Creatures", which could be anything we chose them to be. The class GeneticAlgorithm accepted three functions on these creatures -- getRandomCreature, breed, and fitness. The first, getRandomCreature, generated a random creature, which was used to create the initial "population". Breed took two creatures and produced another creature which was effectively the average of the two (with a healthy dose of randomness, to allow the population to vary). Fitness was a scoring function, which was then used in selecting which creatures to breed and advance to the next generation. By iterating generations of selective breeding, the "creatures" progressively get higher and higher scores, which corresponds to a higher final score.
When deciding the creature, we had a few options. Ultimately, we went with defining a creature as a sequence of waypoints for the robot to hit on the map. This way affecting earlier waypoints wouldn't change the location of later waypoints (as it would have if we had simply used instruction sequences as creatures). Our breeding function averaged waypoints, adding in randomness. Our fitness function made use of an A* pathfinding algorithm to determine the optimal path between waypoints, and the fitness was the score at the end of traversing all waypoints.
It's worth noting that the environment was by no means static, with rocks falling, beards growing, water rising, etc. Because of this, we had to define our A* system to operate on Nodes which were actually a series of MineState objects. Connections between nodes were commands to the robot. Of course, since movement was still grid-based, we were able to use Euclidean distance as a reasonably effective heuristic. The elegance of this system was that our MineState class could then handle all special cases, including ones added as the contest progressed, and our actual algorithm would not be affected at all.

Conclusion:
Though our algorithm had a lot of potential, implementing it using C++ (in order to get the speed we needed) instead of a more functional language was painfully slow. In the end, we ran short on time and weren't able to add in optimizations such as representing the MineState in memory as a series of diffs. Our genetic algorithm ran fairly well on smaller maps, but anything that got remotely larger started to eat up a lot of memory, as each MineState stored it's own copy of the grid, and A* created many copies of MineState.

IPSC 2012 Results

Overall

IPSC this year had an interesting mix of normal algorithm programs and some more unique ones. A breakdown of some of the more interesting problem is given below. Overall, we did much better than last year, finishing in 213th place out of 644 teams who scored points, with a final total of 11 points (7 easy problems and 2 hard ones). With IPSC out of the way, we can set our eyes on ICFP, our all-time favorite competition, but more on that at a later date. Without further ado, our breakdown of the problems:

Problems

A - Abundance of Sweets

As usual, Problem A was not much of a challenge. The easy solution required simply iterating through a matrix and outputting the number of 'o's found. The harder challenge required checking the surrounding values of all the 'o's, but could still be easily done with a brute force method.

B - (blank)

This was one of the more interesting problems of the challenge this year. Given no problem statement, we moved on from this challenge, though in hindsight we should have attacked this problem earlier on. When we did get around to it, we tried a random submission and immediately saw how this problem was laid out when we received a hint in response. Following the hints, we eventually understood what the solution was -- a sequence of 6 or more numbers forming a sequence such that each number was a multiple of the one before, and all of them totaled 123456789. Eventually we managed to use the prime factorization and some educated guessing to come up with a solution that worked. The better solution was much simpler -- just take the binary representation to get a sequence of powers of 2.

The hard counterpart for this problem used a similar hint upon submission method, though required more submissions to move towards the answer. We made good progress on this solution, but unfortunately ran out of time just as we were only 1 or 2 tries away from completion.

C - Card game

This problem dealt with probability, not one of our strong points. After a few wrong tries, we were able to get the first solution, but decided to work on other problems instead of attempting the hard part.

D - Data mania

This problem was more of an aggregate of problems, consisting of several data analysis problems on last year's IPSC user statistics. Each problem in the easy set was relatively easy, and we arrived at an answer, but were judged incorrectly. After looking through it, we were unable to find our mistake, so we moved on.

E - Evil matching

We looked over evil matching some, but didn't come up with any good methods to approach it, and ultimately weren't able to code any solutions.

F - Fair coin toss

This was again a problem focusing more on probability. After thinking through the problem, we came to the correct conclusion for the easy problem that only having a 1/2 probability coin would do it.

For the hard problem it took some thinking through, but it wasn't too hard to come up with a formula for the number of subsets of coins which had at least one probability 1/2 coin in it.

G - Gems in the maze

The easy sub-problem for this problem was easily brute forced. Simply looping through all initial chambers and following the paths was enough to pass.

For the hard test case, we initially though that dynamic programming was the solution, but realized that the possible looping made this impractical. After some thought, we realized that all chamber sequences were some sequence followed by some loop (either of which could be of zero length). In addition, once you found this sequence, all chambers on this sequence were determined, thus giving a linear time algorithm. Implementing this bug-free was somewhat of a challenge, and we ultimately ran out of time on this problem.

H - Harvesting potatoes

For the easy case, we realized that casework would get the job done. Unfortunately as we coded the solution, more and more edge cases came to mind, and the code became complicated quickly. Looking at the scores, we noticed that none of the top teams had finished the problem, so we moved on. At the end we didn't have enough time, and didn't end up finishing either part of the problem.

I - Invert the you-know-what

This was again a fun problem consisting of several username:password hashes which you were tasked with the challenge of decrypting. Luckily the easy subproblem only required decrypting one of the passwords, and passing them through a simple MD5 decrypter got us several passwords. We were stuck for a little bit because the salt prepended to the password threw us off. When we realized to remove the salt then submit we got the easy case.

We weren't sure how to tackle the hard sub-problem, so we moved on.

J - Jukebox

This problem was also a very unique one. As input, you were given MP3 files and the same files in text format, which described waveforms. Unfortunately, we realized analyzing these would require a Fast Fourier Transform, so instead we decided to listen to the pieces and simply transcribe them by ear (at least for the easy case). After a few mistakes, we got a correct submission.

For the hard sub-problem this was clearly not possible, and we didn't have time to figure out writing a Fast Fourier Transform, so we moved on.

K - Keys and locks

In hindsight this problem was very tractable, but we didn't have an good ideas at first and so didn't attempt this problem for very long.

L - Light in a room

After reading this problem, we knew it would take a while and moved on to easier problems.

M - Matrix nightmare

After reading through the wise words at the beginning of the problem statement and quickly scanning through, we again realized this was beyond our level and moved on.

IPSC 2012 is coming up!

The annual Internet Problem Solving Contest, a mathematics and programming competition open to all participants from around the world, is coming up this weekend. We competed in IPSC last year, and unfortunately didn't do as well as we would have liked. This year, Ryan and I are back, with vengeance.

IPSC accepts teams of up to three, so we asked some of our friends at MIT if they wanted to participate. This year we will be participating with Timothy Yang, a rising sophomore at MIT. Tim is also an EECS major, making that all three of us. We're not quite as math-heavy as we might have wanted, but we're still hopeful that something of our first year at MIT will carry over into the competition.

We'll just have to wait until the competition to find out how well we do!

Tej Kanwar
IDKJava

Clower Stats

Our partner in crime for ICFP this year, Mr. Michael Clawar, recently established his new website for the various statistically-oriented projects that he does in his free time (he's crazy). I just thought we'd throw up a link if those of you as crazy as him want to check it out. It's really pretty cool actually.

Here ya go: www.clowerstats.com

ICFP 2011: Results

After many hours of hard work, we finally got the results ... and took place 192 out of 199! Though this may sound terrible, and it is, it is better than last year, in which we tied for last with 0 points because the competition was so devilishly hard. This year, we got a total of 68 points. Considering a win was worth 6 points, and a tie was pretty rare, this probably means we won 11 games and tied 2. There was a total of 198 games, so that's a pretty miserable win/loss ratio. In retrospect, I think our biggest problem was that we targeted the most used slots in hope of breaking them, but we didn't really kill slots fast enough. If we had the time we should have made use of some of the higher functions, especially zombie. Unfortunately our schedules really conflicted with ICFP this year, so we weren't really able to all get together and get a solid few days of work done on it. Overall, I'm pretty happy with the results, because there are some pretty experienced programmers in this competition, and to even have placed above any teams at all is a pretty good accomplishment.

ICFP 2011

It's been a few days since ICFP 2011 and we've finally had a chance to cool off and relax, so I thought I'd throw a blog post up about what we actually did.

Before I go into the details of the competition, allow me to introduce our team. We had four members in total. Three of us are prefrosh at MIT: Ryan Cheu, Kevin King, and myself Gurtej Kanwar. The three of us also happen to be the core members of IDKJava. Our fourth member, a talented statistician named Michael Clawar, is a rising senior at the high school Ryan and I attend. Naturally we were wary, because he aspires to attend our rival, Ha'vard. He was, however, a valuable member of our team, and we grudgingly came to accept his choice. With this formidable team assembled, we were ready to dominate.

We started off at around 8 PM, when the contest opened in our local timezone. We read the problem and immediately felt a little lost. None of us had any experience with Lambda Calculus or SKI combinators, and so we spent much of our precious time figuring out how the problem worked. In addition, we immediately had issues with the fact that we could not all meet up for the majority of this competition, because Kevin lives several hours away from the rest of us, and Michael was out of town for the majority of the competition. As a result we ended up using a lot of Skype, AIM, and Facebook to figure things out.

After finally getting a grasp of the problem statement about a day in, Ryan and I got together and we started work on simulating the situation, so that our "player" would have some idea of what was going on to make AI decisions. Ryan and I met up Friday evening, and we got cracking. We realized that the best way to code this would be to create some of our own data structures, such as a struct Slot and a class Card. One question we ran into was how to represent non-basic functions, such as functions that had been returned by S or attack. We decided to create individual classes for each of the different "stages" in these cards. For example with S, there is the main S card, and then two intermediate stages. Thus we had classes called S1, S2, and S3 to represent this. These classes could then call on the display functions of the arguments they had been passed to display themselves in proper parenthesized form.

Once we had our basic simulation set up, there was about a day's worth of debugging to go through, and special cases to add in. In particular, the zero card threw us off, because originally we thought there was a difference between it and the number 0. In the end, however, we realized that zero and 0 were the same thing, and then zero could not be called as a function as we originally thought. In addition we had to work out the zombie loop and add in another function to deal with the fact that some cards changed how they worked when being called by the zombie loop.

This took up the majority of our time on Saturday, leaving us just Sunday to get some strategy done. On Sunday I set up our program with the framework for creating an AI: a system for adding moves to a queue to be executed (actually a stringstream), and a few utilities, such as clearing slots, applying an SKI combinator, and applying a slot to another slot (given that the first slot was a function). I then unfortunately had to go to work, leaving Ryan to develop our entire strategy for all of Sunday. When I came back with two hours left in the competition, I found an elegant system, in which our program would constantly change which slots it would use, and constantly use help to add vitality and then use that extra vitality to attack the slots most used by the opponent program. Unfortunately, there were several bugs with our system, and it did not work as expected or segfaulted repeatedly, so we spent much of our remaining two hours trying to work out these bugs. With an outside pair of eyes, thankfully we managed to work them all out and with only a few minutes left to go had a working program. Because we were short I did not get a chance to test using the provided utility to see if our program actually worked, but we managed to get everything in on time, and we'll see how we did soon!

I'm pretty proud of how much we accomplished with such sub-optimal timings and conditions. If we had had all four days free and all of us could meet up, I believe we would have been in much better shape, but the fact that we actually submitted a working program at all is an improvement over last year, in which we were close but just missed the deadline on getting in something that worked. We did a great job!

Tegra 2 Issues

Hi Everyone,

We've been fighting a bug recently on the Atrix relating to our textures not working correctly in our development version. We've tracked the problem down to glTexSubImage2D in android native. It appears that the width of the area which you wish to change must be a power of 2. The height however does not need to be a power of 2. This behavior has not been observed on any other device to our knowledge. Hope this helps anyone who has similar issues!

-IDKJava Team

EDIT: We got it working on the first try! After all we've been through with this issue I think we deserved it.