This week, I'll going for something slightly simpler, though it has virtually infinite room for expansion and optization.

Challenge Three

Write a pattern matching engine. Given a pattern and a string, return a boolean match or no match response:

matches = pmatch( pattern, inputstr );

The pattern syntax can be anything other than straight substrings, though you must start by selecting a subset of the PCRE syntax. Start with the '*', '+' and '?' quantifiers, together with character classes and alternations. Ignore capturing, but include grouping. That set of features is probably the minimum set to provide useful matching facilities, but it's not so overwhelming as to require enormous amounts of code. Add to those features as long as it remains interesting.

Some random notes:

• If you primarily work in a 'high level language' like Python or Perl, I suggest trying this in something lower level.
• I also suggest seperating the pattern compilation from the matching, either internally, or explicitly in the API.
• Note that classes can be represented as alternations. '+' can be expanded to a representation using only '*'. '?' can be expressed as an alternation.

It may seem foolish to work at solving a problem so widely and commonly solved, but this is in fact a challenging problem with lots to think about. If you work with regular expressions every day, take this opportunity to look at them from the bottom.

Challenge Two

The Knight's Tour is an old math problem: given a standard chessboard and a single knight, can you move the knight such that you land on every square exactly once. The knight must move as per normal chess rules, you can start and end on any square. A slight variation is a closed tour, where the knight ends up one legal move away from the starting point.

The challenge is to produce a program (in any language) that returns a solution to the puzzle.

None of the following are problem requirements, but worth considering:

• generalize the problem, and allow the size of the chess board to be user specified. Schwenk's Theorem provides some board constraints that are known to affect problem satisfiability. (Short version is that there are solutions if at least one dimension is even, and the board is reasonably large.)
• produce only closed path solutions, or allow the user to select
• produce random solutions. There are known to be billions of solutions for the standard board - program so as to find a solution non-deterministically.
• Wikipedia notes that it is possible to solve the general problem in linear time. It's not required that the solution be optimal, but consideration should likely be given to efficiency.

Some notes, none of which are promised to be helpful, but have occurred to me as I posed the problem:

• The solution set can be represented as a sequence of transition pairs - once all the transitions are selected, then the ordering is unambigious.
• All of the knight's moves for a given square are represented by a single vector, reflected horizontally, vertically or diagonally.

Weekly Programming Challenge

I'm going to gather ideas from a bunch of places, and then write up a challenge. The problems are language neutral, so maybe I'll try some in a few different languages, or different problems in different languages.

There are a few versions of "99 Lisp/Prolog/Whatever Programming Problems" floating around, and I'll take my inspiration for the first few from them.

Challenge One

Goldbach's conjecture states:

Every even integer greater than 2 can be written as the sum of two primes.

Write a function that, given a even integer greater than 2, returns a pair of primes that sums to the given integer.

• Note that the prime sums are not necessarily unique. You can pick a rule for which set to return, but only return one pair. (ie. first found, or the pair with the smallest absolute difference)
• Should raise an exception/etc. if not even, not larger than 2.

Example: (in Perl)

my @result = goldbach( 6 );  # result 3, 3
my @result = goldbach( 10 ); # result 3, 7 -or- 5, 5 

Solutions:

First week, so I need to work out the plan. I'll post my solutions after at least 24 hours, but I'll just link to them without notes or solution details on this page. I'll link to anyone else's solution that requests it, you can also link to your solutions/etc. in comments. I'll defer posting comments that contain too much information until at least after the first 24 hours.

I'll always provide my solutions, but I'll stress that I make no suggestion at all that mine are good or completely correct. In particular, I'm planning on trying these problems in a variety of languages, some of which I am likely to suck at. (The only language I can reasonably claim to not suck in is Perl.)

My Solutions:

• Perl solution: All the program and solution notes are with this code. Solution is only about 9 logical lines of code, perhaps a little too golfed in places.
• C solution: Virtually no documentation, follows the exact same algorithm as the Perl solution.

Other Solutions:

• hapoteh (Perl) - notes inline. Slightly different approach, precomputes primes in range first.

YAPC was good this year. I didn't find it as intellectually rewarding as the previous two, but I don't think that's necessarily a reflection on the conference. I'm just not that interested in AJAX, try as I might, and there were a number of AJAX talks. Also, a lot of the cooler things I have seen in previous years. MJD's talk would have been awesome if I hadn't read his book (I sat in for a while - interesting, but familiar.) As with last year, the best talk were pmichaud's Rule and Parser talks. I missed out on the APL talk, though, because I misunderstood the abstract. Ahh well. Overall, still an awesome week, well worth it. A great break, and a change of scene.

I started working on two projects at YAPC, which I've barely touched since coming back ($work has been quite consuming). They are: converting the guts of the mosaic code to use C via-XS. I figure I can likely get a hundredfold performance boost that way. Learning curve is quite interesting. Like everything over the last week, I've just been reading lightly during downtime.

The other project is to write a Huffman compressor in C++ -- part of my longer term goal to write one in ten different languages. I've done it in Perl5 and Perl6 (as much as possible). Would like to do it in PIR, then perhaps some of: OCaml, Haskell, Ruby, Python, Eiffel, Scheme. Part of me is tempted to toss Fortran in there. It's long past due for me to take a world tour of languages. Huffman compression is interesting, yet simple enough. The Perl5 implementation took an hour or two - so add in the learning curve of a new language, and you've got a good sized project. Solving the same problem multiple times has the upside of allowing direct comparison of approaches. I'll be careful to try to approach the problem fresh each time, so that I'm not writing Perl in Haskell, etc.

Here is a mosaic from a photo I took on Georgian Bay:

Here is a close up of a transition area (sky -> trees -> water):

The repeat limit was manhattan distance of 40, the build order was shuffled. I used the YCbCr colourspace, which I think performs way, way better than HSV/HSL. YCbCr is, I think, the 8-bit version of YUV, and I think that it does a good job of mapping colour perceptually. It gives two components to chroma, rather than HSVs one, which makes for a clean color match. I found that RGB tended to favour blue disproportionately... in fact primaries seemed to match more strongly. The YUV/YCbCr matching gives a softer, more 'real' and slightly more impressionistic feel, I think. I intend to run this again with just RGB, for a straight comparison of the two.

HSL/HSV also represent chroma as a loop... which means that at the 360-0 boundary, the maximum span, there is great similarity. This resulted in poor matches near the boundary. The two dimensional chroma space of YUV/YCbCr place perceptual extremes at the edges, with no wrapping. This means that matches appear good throughout the perceived colour gamut. I'm not a graphics expert in the least, so you'll excuse some heavy misuse of jargon and some similarly heavy glossing.

For those who want the full 4.3MB bomb, here is the full-size version (3750x6450 pixels). Be forewarned, I just selected the images based on license characteristics, I didn't review all 80k tiles for appropriateness, etc.

I think that the fine detail improves considerably as I add to the corpus, but obviously that is RAM limited. Each tile requires an in-RAM 75 element array to represent the vector. It also extends the runtime of the match linearly... this mosaic ran overnight, for about 9 hours.

I haven't gotten my physical copy in the mail yet, though. I also haven't gotten any author email telling me I'm brilliant or that I missed something obvious. So that is half good news, half disappointing.

I've sort of stalled on my Sudoku Grader project... I did complete the Locked Candidate detector, and flesh out the outer framework of the Grader, but I haven't gotten much farther. I might pick it up again in a month or so. I'm trying to follow AudreyT's approach, and be productive in hobby-coding by optimizing for fun. The big downside of that is that when you hit a patch of boring refactoring, you tend to stall.

Update: When I got home, it had arrived. It looks pretty good, I think.

Here's my favourite example so far:

I am most of the way through the HSV chunk that he mentioned... I am including the abilty to weight the HSV so that you can choose which component is more important.

more added later:

Someone on Daniel's blog suggested YUV instead of HSV:

my $y =  0.299 * $r + 0.587 * $g + 0.114 * $b;
my $u = -0.147 * $r - 0.289 * $g + 0.436 * $b;
my $v =  0.615 * $r - 0.515 * $g - 0.100 * $b;

That actually has the additional benefit of being easier to calculate.

I'm pretty torn about my system of weighting dimensions of the colour space. I'm curently scaling my vectors in those dimensions when I create the vectors, rather than weighting differently in the vector distance calculation phase. That makes loads of sense, since it avoids millions of calculations... but the way I have things set up, it applies the scaling at tile index time, rather than index load time. I'm thinking that I should just always index in RGB, then transform the space and scale the vectors during the pre-matching phase (what Daniel and I have been calling phase 2a).

Okay, I'm not torn anymore. That last paragraph untore me.

1. grading and classifying puzzles
2. generalize the exact cover to play around with Kataminoe covers

The first item is first. After sampling just a few of the thousands of puzzles I generated, I noticed a few things:

• A good percentage don't appear to be human-solvable
• I find puzzles most interesting if they are tricky, and contain a known pattern... like a multicolour or an X-Wing.

Both grading and categorizing require something that recognizes patterns, and the last few days, I've been figuring out all of these patterns, and coming up with napkin pseudocode for detecting them. I'm still having a struggle with XY-Wings. Swordfish and X-Wing patterns actually turn out to be easier to find, I think, than hidden pairs. Things like remote pairs and inference chains are extremely difficult for people like me, and programs, to find. You need creepy FPGA brains for that sort of thing.

I already have detectors for Naked Singles, Hidden Singles, and Naked N-Tuples. Next in my agenda is locked candidates. I'm not sure how I am going to implement Colours, yet, though detecting complements is fairly easy.

I'm going to keep posting here as I make progress. Ideally with my basic design next, then progress on each detector.

I'm going to start posting here about programming whatsit that interests me. My degree was in math/computer science, but I wasn't terribly serious about it at the time, to be honest. Lately, certain aspects of computer science have interested me more and more. I've come to a couple of conclusions:

• I like language and symbol aspects, like data and semantic encodings
• I like puzzles, and problem solving with code
• Hardware continues to be boring to me
• I need to learn more languages, try more problems

I'm okay with my preferences and interests. I'll post about them here when I feel like it is interesting enough. I'm going to take on projects, and discuss them with myself here.