Lisp Is Smart Programming

Income inequality analysis and visualization

2018-11-24T00:00:00+00:00

Recently, I did some analysis of the income inequality data found from the world income inequality database and the US government historical income inequality data.

And I was amazed to find some insights through this process. The above is the tableau interactive notebook you can play with. Here I will just list the main findings of the data:

For US government data:

Most age groups’ in US, the median & mean income diverge, and the gap is bigger and bigger as time goes on
For the working males(24 to 65), the median incomes are stagnating since 1970s(inflation adjusted), which means most of the working males don’t see actual income increase through the last 40 years
The top richest people are richer and richer during the same time. For example, the top 5% people’s income percent goes from around 17% in early 1970s to around 22.6% in 2017.
The diverge of the rich and the poor can also be reflected by the gap between mean and median income, which means only a few people are much richer than before while the majority get barely nothing
There is a big jump for the top 5% people from 1992 to 1993(from 18.6% to 21.0%), I then try to find what happened at that time, and it is actually some chaotic period for the Clinton administration, which he want to increase tax for rich class, but actually hurt the middle class.
Although the average income for all ages(15 and older) increase from $2000 in early 1970s to $32,000 in 2017, the actual income(inflation adjusted) only increase from $16,000 to around $34,000 in 2017(only doubled instead 15x times)

And although I also don’t believe the official inflation rate truly reflect the actual number of inflation, but even let us take it as it is, I am stilled shocked by how the actual income of most people are stagnating over the past 40 years. If you want to know more about the inflation information, I recommend you to check the Shadow Statics about alternative measure of different economic indicators like inflation, unemployment, GDP etc.

For other countries, I also find some interesting points:

Within 1991 to 1993, Russia sees a big jump for the GINI index, which is most likely due to the collapse of the Soviet Union.
China was quite equal before the reform and opening(although quite poor at the same time)
Europe is general more equal in income compare to other developed countries
Most countries with high GINI index(greater than 40), the top 20% people will get around 50% income of all people.

Actually, the “richer get richer while poor get poor” can be show in a very simple model in netlogo. In this “simple economy” model, we just need to initialize an amount of people and each with a fixed amount of money. In each round, one person just randomly select another person and give him/her one coin. So this is a extremely simple simulation of the economy system, there is on creation of new things, no money printing etc. But even in this ideal world, we could actually see inequality occurs.

When the system is running and running, we can notice that the top 10 richest people get more and more money while the amount of the bottom 50 are less and less which means some people just accumulate a lot of wealth totally by lucky and the system become less and less equal overtime. It can be shown as in the following image which record the running of the model:

NetLogo Simple Economy Model

And very interesting, the distribution of the wealth change from like a normal distribution at the beginning to right skew and then finally to a exponential like distribution in the late phase. So it means in this really simple economic world, income inequality could happen, and I believe all the thing like Quantitative Easing just aggravate the inequality of the world.

This is an interactive notebook and you can play with the data by selecting different filters. And I find it is normally more useful to use visualization to show interesting phenomena than plain numbers. If you find some other interesting things, you’re welcome to comment after the post :).

Prolog clp(fd) solve killer sudoku and greater killer sudoku

2018-11-18T00:00:00+00:00

Solving standard sudoku using SWI-Prolog clpfd

In the manual of clpfd of SWI-Prolog, there is already an example of using the clpfd to solve the puzzle Sudoku. It really show the power of Prolog’s constraint programming to solve hard problem with a bunch of lines of code. But recently, I find out that there are a lot of variants of Sudoku. So I would like to show how can we add very small modification to the Sudoku solver to solve this problem in Prolog very easily.

Killer Sudoku & Greater Killer Sudoku Problem Definition

Killer Sudoku is a famous variant of standard Sudoku. In additional to the standard Sudoku rules, a Killer Sudoku is divided into cages, the sum of the numbers in a cages must equal the small number in the left corner, and same number can not appear in a cage more than once. The Killer Sudoku is a combination of Sudoku and Kakuro.

Killer Sudoku from Wiki

Greater Killer Sudoku combines the rule of Killer Sudoku and Comparison Sudoku, and has sum relation for some of the cages instead of the sum information. An example of Greater Killer Sudoku can be show as the following image from daily killer sudoku.

Example of Greater Killer Sudoku

Solving Killer Sudoku

To solve the Killer Sudoku, we just need to add the cages’ constraints to the solver, when we define the problem, we also give the cages’ information and their corresponding sum.

The whole program is something like the following.

get_value_by_key([], _, _) :- !.
get_value_by_key([X-Y|_], X, Y) :- !.
get_value_by_key([Z-_|Ps], X, Y) :- Z \= X, get_value_by_key(Ps, X, Y).

region_constrain(Sums, X-Vs) :-
    get_value_by_key(Sums, X, Value),
    all_distinct(Vs),
    sum(Vs, #=, Value).
    
killer_sudoku(Rows, Splits, Sums) :-
    length(Rows, 9), maplist(same_length(Rows), Rows),
    length(Splits, 9), maplist(same_length(Splits), Splits),
    append(Rows, Vs), Vs ins 1..9,
    append(Splits, Ks),
    pairs_keys_values(Pairs, Ks, Vs),
    sort(1, @>=, Pairs, SortedPairs),
    group_pairs_by_key(SortedPairs, Regions),
    maplist(all_distinct, Rows),
    transpose(Rows, Columns),
    maplist(all_distinct, Columns),
    Rows = [As,Bs,Cs,Ds,Es,Fs,Gs,Hs,Is],
    blocks(As, Bs, Cs),blocks(Ds, Es, Fs),blocks(Gs, Hs, Is),
    maplist(region_constrain(Sums), Regions).

So we only need to pass the additional cages’ information and their corresponding sum information, and build the constraint relation for them. It is only around 10 lines more code than the standard Sudoku solver, and we can already solve every killer sudoku in Prolog now.

And also we should notice, despite it’s name, Killer Sudoku’s difficult levels varies, simple Killer Sudoku may even be simpler than normal Sudoku, but the hardest one may take hours even days to crack.

So I do some test to solve various Killer Sudoku I found from online, and see how well the solver can tackle the problems.

For the problem from wiki, it solve the problem instantly, but for some harder problems, it may take minutes even hours to solve.

?- problem(wiki, Rows, Splits, Sums), killer_sudoku(Rows, Splits, Sums), 
|   append(Rows, Vs), time(label(Vs)), maplist(portray_clause, Rows).
% 591 inferences, 0.000 CPU in 0.000 seconds (98% CPU, 17776040 Lips)
[2, 1, 5, 6, 4, 7, 3, 9, 8].
[3, 6, 8, 9, 5, 2, 1, 7, 4].
[7, 9, 4, 3, 8, 1, 6, 5, 2].
[5, 8, 6, 2, 7, 4, 9, 3, 1].
[1, 4, 2, 5, 9, 3, 8, 6, 7].
[9, 7, 3, 8, 1, 6, 4, 2, 5].
[8, 2, 1, 7, 3, 9, 5, 4, 6].
[6, 5, 9, 4, 2, 8, 7, 1, 3].
[4, 3, 7, 1, 6, 5, 2, 8, 9].

Difficulty level 10/10 Killer Sudoku from the daily killer sudoku website. It toke around 2 minutes to find the solution, and take around 6 minutes to prove that there is on other solutions. Since it is marked as the most difficult Killer Sudoku on the website, I think the speed is already quite fast. On the website, it said the average time to solve the problem is 44 minutes, but it may because you can use hint for the task for simple problem(difficult level 1/10), the average time is also around 40 minutes, so it is likely either all the people who try to solve the difficult one are expert at solving Killer Sudoku or the time is not actually reflected the difficult level of the problem(like people want to use hint when they stuck).

10/10 Killer Sudoku

?- problem(daily_killer_sudoku_hard, Rows, Splits, Sums), killer_sudoku(Rows, Splits, Sums), 
|   append(Rows, Vs), time(label(Vs)), maplist(portray_clause, Rows).
% 277,532,279 inferences, 112.211 CPU in 112.575 seconds (100% CPU, 2473301 Lips)
[2, 9, 6, 7, 1, 3, 4, 8, 5].
[3, 7, 5, 2, 4, 8, 1, 9, 6].
[1, 4, 8, 5, 6, 9, 3, 7, 2].
[8, 5, 3, 9, 7, 1, 2, 6, 4].
[9, 6, 1, 3, 2, 4, 7, 5, 8].
[4, 2, 7, 8, 5, 6, 9, 1, 3].
[6, 3, 2, 1, 9, 5, 8, 4, 7].
[5, 8, 9, 4, 3, 7, 6, 2, 1].
[7, 1, 4, 6, 8, 2, 5, 3, 9].
% 906,040,567 inferences, 382.551 CPU in 384.761 seconds (99% CPU, 2368417 Lips)
false.

Then I coincidentally encountered a page said that there is a most difficult 19 cages Killer Sudoku in the world. Although I do not think there is a objective standard for difficulty, but it seems to be quite difficult since every cage is not limited to one block which make simple strategy for solving Killer Sudoku not working. And in the post, the author said it take him six days to solve the problem. So I decide to give it a try to the constraint solver.

Most difficult killer sudoku

It actually take quite long to solve the problem(around 12h to get the solution). But since except the definition constraints of the problem, I do not use any other advanced strategy, I think the speed is already satisfactory.

?- problem(most_difficult_killer_sudoku, Rows, Splits, Sums), killer_sudoku(Rows, Splits, Sums), 
|    append(Rows, Vs), time(label(Vs)), maplist(portray_clause, Rows).
% 98,413,615,883 inferences, 43078.045 CPU in 43212.116 seconds (100% CPU, 2284542 Lips)
[9, 2, 8, 3, 4, 7, 6, 1, 5].
[3, 6, 7, 5, 1, 8, 4, 9, 2].
[5, 1, 4, 6, 2, 9, 7, 3, 8].
[2, 8, 3, 9, 7, 6, 1, 5, 4].
[7, 9, 1, 4, 3, 5, 2, 8, 6].
[4, 5, 6, 1, 8, 2, 9, 7, 3].
[6, 3, 2, 7, 5, 1, 8, 4, 9].
[8, 7, 5, 2, 9, 4, 3, 6, 1].
[1, 4, 9, 8, 6, 3, 5, 2, 7].

Solving Greater Killer Sudoku

To solve the Greater Killer Sudoku, we just need to add the comparison constraint relation to the solver. And we should also notice that some of cages do not have sum information, so when build the constraint for sum information, we should only select the area with sum information. The program is like following.

compare_constrain(Regions, [X, Y]-Relation) :-
    get_value_by_key(Regions, X, Xs),
    get_value_by_key(Regions, Y, Ys),
    all_distinct(Xs), all_distinct(Ys),
    sum(Xs, #=, Vx),
    sum(Ys, #=, Vy),
    (Relation = less
    ->  Vx #< Vy
    ;   ( Relation = equal
        -> Vx #= Vy
        ;  Vx #> Vy
        )
    ).

has_sum_info(Pos, X-_) :- member(X, Pos).

greater_killer_sudoku(Rows, Splits, Sums, Compares) :-
    length(Rows, 9), maplist(same_length(Rows), Rows),
    length(Splits, 9), maplist(same_length(Splits), Splits),
    append(Rows, Vs), Vs ins 1..9,
    append(Splits, Ks),
    pairs_keys_values(Pairs, Ks, Vs),
    sort(1, @>=, Pairs, SortedPairs),
    group_pairs_by_key(SortedPairs, Regions),
    maplist(all_distinct, Rows),
    transpose(Rows, Columns),
    maplist(all_distinct, Columns),
    Rows = [As,Bs,Cs,Ds,Es,Fs,Gs,Hs,Is],
    blocks(As, Bs, Cs),blocks(Ds, Es, Fs),blocks(Gs, Hs, Is),
    maplist(compare_constrain(Regions), Compares),
    pairs_keys(Sums, PosHasSumValue),
    pairs_values(Regions, Vss),
    maplist(all_distinct, Vss),
    include(has_sum_info(PosHasSumValue), Regions, RegionsHasSum),
    maplist(region_constrain(Sums), RegionsHasSum).

We define another predicate called compare_constrain which take the regions information and two cages’ comparison relation. It is something like [a, b]-less which means cage a’s sum is less than cage b’s sum. And we just add this constraints to the solver. All the remaining are exactly the same as Killer Sudoku’s solver.

I use the above solver to solve the most difficult greater killer sudoku from the daily killer sudoku as following, and it take much more time to solve the problem (around 5 hours) than the most difficult Killer Sudoku from the website. I think that’s because the compare relations are not very strong constraint and some cages missing the sum constraint information which make the search space for the constraint propagation much much larger than normal Killer Sudoku.

Example of Greater Killer Sudoku

?- problem(daily_greater_killer_sudoku_hard, Rows, Splits, Sums),
|     greater_killer_sudoku(Rows, Splits, Sums,
|        [[a,b]-equal, [j, o]-less, [r, v]-equal, [u, y]-greater, [y, v]-equal]),
|     append(Rows, Vs), time(label(Vs)), maplist(portray_clause, Rows). 
% 26,391,594,689 inferences, 17021.320 CPU in 17080.693 seconds (100% CPU, 1550502 Lips)
[5, 4, 2, 9, 6, 8, 7, 3, 1].
[8, 9, 1, 3, 7, 5, 2, 4, 6].
[6, 7, 3, 4, 2, 1, 5, 8, 9].
[9, 2, 8, 5, 4, 7, 1, 6, 3].
[3, 1, 7, 8, 9, 6, 4, 5, 2].
[4, 6, 5, 2, 1, 3, 9, 7, 8].
[7, 3, 6, 1, 5, 2, 8, 9, 4].
[1, 8, 4, 7, 3, 9, 6, 2, 5].
[2, 5, 9, 6, 8, 4, 3, 1, 7].

Conclusion

So the intention of this post is not to build the fastest Killer Sudoku and Greater Killer Sudoku solver, just want to show how we can just add a bunch of lines of code to the standard Sudoku solver in clpfd to get a general solver which can solve Killer Sudoku and Greater Killer Sudoku in a quite reasonable time and to show that clpfd in SWI-Prolog is actually a very good domain specific language at solving such puzzles over integer domain. And the expressiveness of clpfd make the solution very straightforward and less bug prone.

So in general, with only around 20 lines of more code, we can solve two famous extension of normal Sudoku problem in Prolog, and the code is concise and express high level definition of the problem and thus is less bug prone, and can actually solve very difficult problems in a reasonable time :).

You can find the whole program from clpfd-killer-sudoku.

Story behind OMail: An email client with automatic mailing list management on Android

2018-10-20T00:00:00+00:00

I still remember the old days when Opera browser is a kind of Emacs in browser world. You have a mail client integrated which you can manage your emails, there is a IRC client which you can chat over different chat rooms. Also the Emacs fans can use Emacs shortcuts in Opera, which is great if you are a user of Emacs editor, since most of your favorite shortcut works out of box, and you can use it either in browser, email editing or IRC chat, and you don’t need to install any extra plugin for that.

Opera Browser add multiple type of accounts

Time flies, Opera change to follow the Chrome browser kernel after version 12. It also make the email client a standalone program, but sadly they also stop the support of this great software. I would not argue whether it is correct or wise decision for Opera to follow this path, since in my opinion, popularity is neither good or bad, it is just popularity. As a pioneer in browser software industry, Opera left us a lot of innovations like tab view, speed dial, mouse gesture etc. And unlike other company, which use patents to make law suit against others, Opera only use patent as a protection for themselves, but do not abuse patents to stop innovation. Opera may fail one day, but all these are the valuable assets Opera left for the world.

Although Opera Mail has this and that problems for opera mails, but I still like this software so much that I wish they could continue to improve it. So for example, the following is an example a lot of mailing lists with only a random number as prefix but are actually the same one.

Mailing list with random number

Although this is quite a easy problem to fix, I also report this to them but got no reply. And it seems that the software is indeed dead in his life cycle. Sigh :(.

As a long time Blackberry fan, I also tried the the mail client on the Blackberry OS 10 system, I would say it is quite a excellent platform and email client, every time when message get into my inbox, it is pushed to my blackberry client almost instantly. I think it is really good for people who wish to stay online forever, and would like to get notification as soon as possible, but it is just not my flavor. And Blackberry is smirking at the iPhone X’s “there’s never been a better way” to interact with your phone. At least it is there back to 2013 when Blackberry release the Q10 device. Not to mention all later innovations like keyboard as track pad, swipe to select and delete text in other Blackberry devices. As Opera, Blackberry also almost fails in the mobile world, but it also does not mean either their product or software are bad or not, maybe just a lack of lucky.

Blackberry Q10 gestures.

But here is the good part to be as a programmer, if you really want something, but there is nothing meet your needs, you can try to build it yourself. That’s why during the last several months, I try to learn and build an email client on Android which do have the similar features as Opera Mail.

It is my first Android application, and I have mixed feelings during the adventure. I have never develop any app previously. At beginning, I want to use some cross platform approach like Xamarin to do this project, since it seems to be promising to just write one software but can run on multiple platforms. But I found that the Xamarin is extremely buggy and most of my time are spending on searching the strange error message just because I update the framework. And even the sample projects offered with Visual Studio does not work. In the end, the Xamarin is like an white elephant to me for developing application. It may have some showcases to build some toy like project which can run in multiple platforms, but if you want to build any real project, it will give you a lot of troubles.

Then I just decided to stick to the Android platform. Although it also has problem and bugs behind, but the experience is much much more smooth than Xamarin. So I started learning the basic block of building android application. Start to do the project bit by bit, solve any problem during this procedure.

After months of work in my spare time, I am proud to announce the initial release of OMail and OMail Pro. It is an email client on Android which have the some of the most important feature as Opera Mail, like

Automatically mailing list management
Account View
Attachment View
Customized control of what to sync for your email on your phone
etc.

It does not have the feature like customized tagging in Opera Mail, because the custom tags can only store locally, but I would like everything can be synchronized between all email clients. So I implement star as the only tagging method for message since it is supported by most of the mail providers.

It also do not do full sync like what Opera Mail do in the desktop, but does incremental sync, which may let some old emails in different state(like you move a very old message to another folder). But it offer the user customized sync which can let the user to sync message during an time range.

The OMail & OMail Pro also will prefer to save plain text email instead of save all mail information like what Opera Mail did. That is because I want to limit the use of storage on a mobile device and also plain text are enough for most programmer is the group who use mailing list the most, and they normally prefer to discuss in an plain text format(Take how popular markdown is nowadays into consideration). And also most html mail are promotion related.

If you’re programmer or any professional work with large amount of mailing lists, and you’re tired of add more and more filters in your email account every time you find you miss some filter, you’re welcome to give OMail or OMail Pro a try, and also you’re welcome to give feedback for any problem or future improvements of these application.

Automatic dataset preliminary analysis in Mathematica

2018-02-14T00:00:00+00:00

Nowadays, we live in an era of information explosion. And from time to time, we may need to do some some analysis of these data. Luckily, we have tools like Tableau or Qlik for analyzing and visualizing these data, but still it is time consuming and even tedious to dig into the dataset to find out what is worth showing in the visualization.

Be a programmer, I’d like to automate as much as I can in any task(or in other words I am lazy). So I decide to automate this process, thus I build a dataset automatic preliminary analysis tool to help me explore dataset.

Here are some screenshots:

The dataset can be find here. The tool can basic numerical, category, correlation visualization, and it can also do cluster analysis in two and three dimensional space as show in the screenshot.

But it can also do analysis on time series data, as show in the following screenshots.

From these visualization, we can see that since 1994, all type of criminals in US steadily drop. While in the food price example, although generally it seems that all price has a quite high positive correlation, but in some specific time period, some price may have different trends, like in the early 90s, the meat price drop significantly while the oil price increase, same for meat price and sugar price from 1993 to 1995.

Not only that, it also support geographic entities like cities, countries etc. For example, the following image show the analysis of cities with more than 500,000 people in US, and do a plot on the map.

It is very clear from the visualization that the mid north part of US has a a much higher rate of murder like criminal, while the south east part of US like California has much higher rate of motor vehicle theft.

This tool does not intend to eliminate all kind of work in exploring a dataset, something like data cleaning, make the dataset a proper format is still need to be done by people. But at least it can help me to quick explore the dataset and help me to identify what need some further analysis for more deep insight.

The functional zipper structure

2017-12-17T00:00:00+00:00

There is a problem called tree manager in HackerRank functional domain . In shorts, there is a tree structure, which has sibling and parent, and there are different operations like visitParent, visitChild, go to left sibling etc which change the current focus of the root node in the tree.

It is clear that this problem can be solved using a imperative data structure, we can use point to point to the parent of current node, and store the children as a array or linked-list for current node. For most operation in the the problem ,it will require $O(1)$ time complexity, only for operation like visitChild(n) ,insertLeft(x), insertRight(x), visitChild(n) which may be proportional to the length of the children depends on the data structure you use to store the children. But since the maximum length of children of a node is 10, it would be enough to solve this problem in the time requirements.

The problem is how can we solve this in an idiom functional way. One easy way is just to use some mutable structure like in imperative language. Another way is to store the structure as a tree, and every update operation we change the whole path from the current focus to the root what we normally do in a persistent data structure. But this approach has a problem, if the tree degenerate into a linked list, every update operation will require $O(n)$ , where $n$ is the length from the root to the current focus, which will cause the worst case of this problem to become $O(Q^2)$ , which will definitively not be able to pass in the worst case.

As the following image shows, if we change the node of $6$ to $6'$ in the tree, although we do not change the value of $1$ and $3$ in the tree, but to make the tree persistent, we need to create all the ancestors node from $6$ to the root to create a new path. In the image, all the circle node and the red paths need to be maintained during the update operation.

The solution is a very elegant functional structure I learned during solving this problem which is I want to share in the post.

The structure is called zipper, which can be use in one dimensional list or in a tree structure. For how to use the zipper in a tree structure, there is a paper which describe how to use the structure in a tree. In this post, I will try to give a more detailed and intuitive explanation of the structure.

Suppose we have a structure, in this structure we have a current focus(you can imagine this focus as a pointer point to some node in this structure).

For example, in one dimension, you can imagine the structure as an array or list , and there is a pointer point to some position in the list, and we can change the current focus to the left, right, add/insert items to the left, right etc.

In a tree form, we can think this as a directory tree in file system, we can go parent directory, go to nth child of current directory, and the current focus is just the current directory at any time.

Let us think first how to solve this problem in one dimension in a functional way. The naive idea is to use a linked list to store the item, and hold the current focus. This will make all operations to the right like switch focus to the right and add item to the current right in $O(1)$ time. But to add item to the left, or switch focus to the left would require copy the node from the beginning to the current node or traverse from the beginning to the current node as show in the following picture, which will make these operations proportional to the depth of current node in the list. In the picture, the current focus is at node 6, if we want to insert a 5 as its left sibling, we need to copy the whole path from the beginning to the current node.

How can we use a efficient functional structure to do all these operations? The solution is amazingly simple, we store the structure in a tuple, the first element is a list of the left sibling of current focus in reverse order, the second item is the current focus, while the third item is the right sibling. For example, the above example will be something like ([4; 3; 2; 1], 6, [7; 8]). And this is also how the name zipper came from, because the structure looks like a zipper. For example, if we continue apply goLeft operation to the structure, it will become ([3; 2; 1],4,[6; 7; 8]), ([2; 1],3, [4;6;7;8]), ([1],2,[3; 4; 6; 7; 8]) and finally become ([],1,[2; 3; 4; 6; 7; 8]), something like an zipper go from open to closed status.

All the operations are show as the following code snipe.

(* left_reverse * current * right *)
type 'a zipper = Zipper of ('a list) * 'a * ('a list)

let goLeft = function
    | Zipper([], _, _) -> failwith "left of leftmost element"
    | Zipper(l :: ls, y, rs) -> Zipper(ls, l, y :: rs)

let goRight = function
    | Zipper(_, _, []) -> failwith "right of rightmost element"
    | Zipper(ls, x, y :: rs) -> Zipper(x :: ls, y, rs)

let insertLeft y = function
    | Zipper(ls, x, rs) -> Zipper(y :: ls, x, rs)

let insertRight y = function
    | Zipper(ls, x, rs) -> Zipper(ls, x, y :: rs)

We defined the zipper as a some kind of a 3 elements tuple, as said above, the first element store the left siblings of current focus in reverse order, the second element is the focus itself, and the right element is the right siblings. Since we know that in functional language, it is very cheap to do operation at the header of list, so all the above operations are $O(1)$ in complexity.

To use this structure in a tree described in the problem is very simple, The only additional information we need to maintain are the parent relation in tree and the sons of the current node, as defined in the following type definition.

(* [left(reversed)] * parent * [right] * value * [children] *)
type 'a path =
    | Top
    | PathNode of ('a path list) * ('a path)  * ('a path list) * 'a * ('a path list)

As the above definition, a path is either a Top or a five elements tuple which hold the reverse left siblings, the parent, the right siblings, the current value and the sons of current node.

And all the operations are quite similar to the linear zipper. The only operations which is not $O(1)$ is goUp, nth and delete as show bellow, since they either need to go through a list to find the nth element or re-construct the information of children nodes of the parents because we use a zipper structure to store them.

let goUp p =
    match p with
    | Top -> failwith "up of top element"
    | PathNode(left, up, right, _, _) ->
        match up with
        | Top -> failwith "up with top"
        | PathNode(left', up', right', value, _) ->
            PathNode(left', up', right', value, (List.rev left) @ (p :: right))

let rec nth p = function
    | 1 -> goDown p
    | n when n > 0 -> goRight (nth p (n - 1))
    | _ -> failwith "n must be greater or equal to zero"

let delete p =
    match p with
    | Top -> failwith "delete of path top"
    | PathNode(left, up, right, _, _) ->
        match up with
        | Top -> failwith "delete the root node"
        | PathNode(left', up', right', curValue, _) ->
            PathNode(left', up', right', curValue, (List.rev left) @ right)

But since depend on the structure we use in a imperative way, we also has several operations which require $O(length)$ time complexity, where length is the number of children node of one element. We already achieve the same time complexity as we can do in a imperative way.

Actually, the zipper structure is very similar to the functional queue, which also split the structure into two part, and also need to reverse part of the structure. The dequeue operation’s worst case is $O(n)$ , but it can guarantee that each operation take only $O(1)$ amortized time.

For example, the following two images are the imperative & functional queues doing the same operation sequence enqueue(1), enqueue(2), enqueue(3), dequeue(), dequeue(), enqueue(4), dequeue(), dequeue(). The functional queue use two list to maintain the head & tail part of the queue, normally the operation only happen at the head of one list, only when the head list is empty and we do a dequeue() operation, we need to reverse the tail part to become the head part, but since every element will at most reverse once from the tail part to the head part, it is easy to prove that the amortized complexity is $O(1)$ . And for the imperative queue, since it can have random access of element in $O(1)$ time, so all the operations are just in constant time complexity.

Imperative Queue	Functional Queue

Final thought

Although I admit that functional programming is more easy to reason the correctness and generally the program is more concise and elegant, but I still think that every paradigm has its own strength and weakness. For this specific example, I think it would be much easier to implement this problem in an imperative way instead of design the structure from scratch(if someone has never be shown any of the functional queue structure or zipper structure, I would doubt the probability of coming out this idea by himself). That’s why I really appreciated language like F# since it allow you to use multiple paradigms in one language, and will have strengths from different program paradigms.

You can view the whole program at hackerrank/Tree-Manager.

Nonogram prolog clpb solution

2016-09-22T00:00:00+00:00

Recently, I was introduced the nonogram problem by some of my friends. And it seems to be another example where constraint programming really shines. So in this post, I will show you how you could solve the problem using SWI prolog’s clpb library in about 50 lines of code.

Problem

The problem statement is really simple, you have a board to fill in black or white, and the problem give the constraints of each row and each columns, based on the constraints, you need to figure out how to fill the board.

The constraints is just the numbers of continuous black in each row and columns.

Example from wiki

Solution

Based on the problem statement, we first need to build the constraints of the problem. At first, it is not very clear how can we build the constraints of the problem. But after some search on the internet, I came across a solution in another constraint programming framework Gecode, which build the constraints using regular expression.

For example, if we use 0 as false and 1 as true, and the constraint is [3, 2, 2], the row as a string should be in regular expression form like 0*1{3}0+1{2}0+1{2}0*.

So I want to follow the same approach to solve it in Prolog. Prolog’s definite clause grammar(DCG) make it really easy to build a small regular expression engine. The following code is of some minor modification of this stackoverflow answer. For users who are not familiar with Prolog’s DCG, please refer to some tutorial here.

:- op(100, xf, *).
:- op(100, xf, +).
:- op(100, xfy, **).

regex(C)                --> [C].
regex([T|Ts])           --> regex(T), regex(Ts).
regex([])               --> [].
regex(eps)              --> [].
regex(_*)               --> [].
regex(R*)               --> regex(R), regex(R*).

regex(R+)               --> regex(R), regex(R*).

regex((R1|R2))          --> ( regex(R1) ; regex(R2) ).

regex(range(R,N,M))     -->
    {between(N,M,L),
     length(D,L),
     maplist(copy_term(R),D)
    }, regex(D).

regex(repeat(R, N)) -->
    length(D, N),
    maplist(copy_term(R), D),
    regex(D).

regex(R**N) --> regex(range(R, N, N)).

Use the above code, we can match a list with a regular expression like

?- phrase(regex([0*,1**3, 0+, 1**2, 0+]), [1, 1, 1, 0, 1, 1]).
false.

?- phrase(regex([0*,1**3, 0+, 1**2, 0+]), [1, 1, 1, 0, 1, 1, 0]).
true ;
false.

Or generate all sequences which satisfy the regular expression like

?- use_module(library(clpb)).
true.

?- length(Row, 8), phrase(regex([0*, 1**3, 0+, 1**2, 0+]), Row), labeling(Row).
Row = [1, 1, 1, 0, 1, 1, 0, 0] ;
Row = [1, 1, 1, 0, 0, 1, 1, 0] ;
Row = [0, 1, 1, 1, 0, 1, 1, 0] ;
false.

Now we can build the constraints construction of the problem. The following code will change one row or column’s constraint to the regular expression form.

constraint_2_regex_tail([], [0*]) :- !.
constraint_2_regex_tail([H|T], [0+, 1**H | TRes]) :-
    constraint_2_regex_tail(T, TRes).

constraint_2_regex([], [0*]) :- !.
constraint_2_regex([X], [0*, 1**X, 0*]) :- !.
constraint_2_regex([H|T], [0*, 1**H| RegexTail]) :-
    constraint_2_regex_tail(T, RegexTail).

Using the previous [3,2,2] as an example, the constraint_2_regex will generate something like

?- constraint_2_regex([3, 2, 2], Res).
Res = [0*, 1**3, + 0, 1**2, + 0, 1**2, 0*].

I don’t know why there is some problem of display 0+ as + 0, but it is actually the correct regular expression.

?- constraint_2_regex([3, 2, 2], Regex), length(Row, 9), phrase(regex(Regex), Row),
|    labeling(Row), writeln(Row).
[1,1,1,0,1,1,0,1,1]
Regex = [0*, 1**3, + 0, 1**2, + 0, 1**2, 0*],
Row = [1, 1, 1, 0, 1, 1, 0, 1, 1] ;
false.

?- constraint_2_regex([3, 2, 2], Regex), length(Row, 10), phrase(regex(Regex), Row),
|    labeling(Row), writeln(Row).
[1,1,1,0,1,1,0,1,1,0]
Regex = [0*, 1**3, + 0, 1**2, + 0, 1**2, 0*],
Row = [1, 1, 1, 0, 1, 1, 0, 1, 1|...] ;
[1,1,1,0,1,1,0,0,1,1]
Regex = [0*, 1**3, + 0, 1**2, + 0, 1**2, 0*],
Row = [1, 1, 1, 0, 1, 1, 0, 0, 1|...] ;
[1,1,1,0,0,1,1,0,1,1]
Regex = [0*, 1**3, + 0, 1**2, + 0, 1**2, 0*],
Row = [1, 1, 1, 0, 0, 1, 1, 0, 1|...] ;
[0,1,1,1,0,1,1,0,1,1]
Regex = [0*, 1**3, + 0, 1**2, + 0, 1**2, 0*],
Row = [0, 1, 1, 1, 0, 1, 1, 0, 1|...] ;
false.

Finally, here are the part to solve the nonogram.

sat_row(Row, Cs) :-
    constraint_2_regex(Cs, RegCs),
    phrase(regex(RegCs), Row),
    labeling(Row).

same_col_length([], _).
same_col_length([H|T], N) :-
    length(H, N),
    same_col_length(T, N).

nonogram(Rows, RowCs, ColCs) :-
    length(RowCs, RowLen), length(Rows, RowLen),
    length(ColCs, ColLen), same_col_length(Rows, ColLen),
    maplist(constraint_2_regex, RowCs, RowCsRegex),
    maplist(constraint_2_regex, ColCs, ColCsRegex),
    transpose(Rows, Cols),
    maplist(sat_row, Rows, RowCsRegex),
    maplist(sat_row, Cols, ColCsRegex).

And here are some simple test cases.

test(1, _,
     [[], [3], [1, 1], [3], [1]],
     [[], [], [4], [1, 1], [3]]).

test(2, _,
     [[4], [1, 1], [4], [1], [5], [1]],
     [[], [5], [1, 1, 1], [1, 1, 2], [3, 1], [1]]).

test(3, _,
     [[], [4], [6], [2, 2], [2, 2], [6], [4], [2], [2], [2], []],
     [[], [9], [9], [2, 2], [2, 2], [4], [4], []]).

You can test it like

?- test(N, Rows, RowCs, ColCs), nonogram(Rows, RowCs, ColCs), maplist(writeln, Rows).
[0,0,0,0,0]
[0,0,1,1,1]
[0,0,1,0,1]
[0,0,1,1,1]
[0,0,1,0,0]
N = 1,
Rows = [[0, 0, 0, 0, 0], [0, 0, 1, 1, 1], [0, 0, 1, 0, 1], [0, 0, 1, 1, 1], [0, 0, 1, 0|...]],
RowCs = [[], [3], [1, 1], [3], [1]],
ColCs = [[], [], [4], [1, 1], [3]] ;
[0,1,1,1,1,0]
[0,1,0,0,1,0]
[0,1,1,1,1,0]
[0,1,0,0,0,0]
[0,1,1,1,1,1]
[0,0,0,1,0,0]
N = 2,
Rows = [[0, 1, 1, 1, 1, 0], [0, 1, 0, 0, 1, 0], [0, 1, 1, 1, 1, 0], [0, 1, 0, 0, 0|...]\
[0, 1, 1, 1|...], [0, 0, 0|...]],
RowCs = [[4], [1, 1], [4], [1], [5], [1]],
ColCs = [[], [5], [1, 1, 1], [1, 1, 2], [3, 1], [1]] ;

Some though on efficiency

Although it is really elegant to solve this problem in SWI-Prolog’s clpb library, the problem is not extremely efficient. For example, the third test will take around 10 minutes to find several answers on my desktop with an I7 processor. But since this problem is an NP problem, it is not that bad(the third problem should be of time complexity of 2^88). But we should know that in practice,

Many nonogram problems can be solved efficiently, because the interrelated constraints on the two axes allow the search space to be bounded, dramatically reducing the space that must be searched for a solution. Nonogram Wiki

And if anyone knows more efficient way to solve this in Prolog’s clpb. you are welcome to tell me :).

Here I don’t intent to make the program the most efficient to solve the problem, I just want to show how elegant the problem can be solved in the constraint programming paradigm.

You can find the whole program on clpb-nonogram.

Using CLP(FD) to solve factorial and fibonacci problems reversely

2016-01-01T00:00:00+00:00

Running program backward in logic programming

One interesting features of logic programming languages like Prolog is that it can run some programs backward, which means that given the output of the program, it can yields all the input satisfies the relation. One famous example is the append function of list.

my_append([], Ys, Ys).
my_append([X|Xs], Ys, [X|Zs]) :- my_append(Xs, Ys, Zs).

The above prolog program give the relation that my_append(Xs, Ys, Zs) which means that list Xs append list Ys the result is list Zs. In a functional languages, the append function is normally given two list and return the result of the concatenation of them. But with automatic backtracking of logic programming, we can actually call the relation with the output, and get all the input. For example, the following is calling my_append(Xs, Ys, [1,2,3,4]) which means for what Xs and Ys, their concatenation is list [1,2,3,4].

?- my_append(Xs, Ys, [1,2,3,4]).
Xs = [],
Ys = [1, 2, 3, 4] ;
Xs = [1],
Ys = [2, 3, 4] ;
Xs = [1, 2],
Ys = [3, 4] ;
Xs = [1, 2, 3],
Ys = [4] ;
Xs = [1, 2, 3, 4],
Ys = [] ;
false.

We can see that the program automatically return us all the possible answer for Xs and Ys whose concatenation is [1,2,3,4].

The magic behind the sceen is the automatica backtracking search of logic programming languages through unification algorithm which was first investigated by John Alan Robinson, so it is also called Robison Unification algorithm.

search tree of append

The above picture shows the search tree of the append, each leaf node is a solution of my_append clauses. Notic in each intermediate node there are two ways to do the unification for each of the my_append clause respectively.

Question about run factorial backward

Over the past few days, there are some people on Quora asked me how to solve the problem of X! = N in prolog. Although prolog has the ability to run program backward, sadly, logic programming can not be used directly in this scenario since the program of fact as follow do use the value of N in the body.

fact(0, 1).
fact(N, F) :- 
    N >= 1, N1 is N - 1,
    fact(N1, F1),
    F is N * F1.

?- fact(5, F).
F = 120 ;
false.

?- fact(7, F).
F = 5040 ;
false.

?- fact(N, 120).
ERROR: >=/2: Arguments are not sufficiently instantiated
?- 

The above example shows that when we call fact(N, 120), the prolog system will complain that >= arguments are not sufficiently instantiated. It is because that we actually use the value which is not given in the call.

Constraint programming to rescue

At first, I try to solve it in pure prolog. So I try to use a binary search algorithm to solve the problem. But late on, I remeber the constraint logic programming model I recently explored, so I decided to give it a try.

In short, constraint programming is a programming paradigm which is even more high level programming model than functional and logic programming. In this model, programmer just need to state the constrains the variable need to satisfy instead of writing the whole algorithm. The constraint solver will try to solve the problem automatically.

Based on the domain, constraint programming normally has the the following categories:

Finite Domain : such as integers, set etc
Real domain : such as linear optimization
Boolean domain : deal with boolean value such as SAT
Tree domain

Since the holistic discussion is beyond this post, readers who are interested in constraint programming are encourage to read Peter J. Stucky’s book “Programming with constrain: An introduction”.

Here I just show you how use “clp(fd)” library in swi-prolog to make a general relation of factorial in prolog.

:- use_module(library(clpfd)).

n_factorial(0, 1).
n_factorial(N, F) :-
    N #> 0, N1 #= N - 1, F #= N * F1,
    n_factorial(N1, F1).

Some calling example of n_factorial as follow:

?- n_factorial(5, F).
F = 120 ;
false.

?- n_factorial(10, F).
F = 3628800 ;
false.

?- n_factorial(N, 120).
N = 5 ;
false.

?- n_factorial(N, 720).
N = 6 ;
false.

?- n_factorial(N, 721).
false.

Facing the problem of non-termination

If you’re a scrupulous programmer, you will notice that the constraint version has a difference with the normal version, it reverse the last two clauses at last. It is done deliberately to avoid non-termination problem. If we reorder the last two clause to the same order as in fact, we get the program as follow:

:- use_module(library(clpfd)).

% non termination after reordering.
n_factorial2(0, 1).
n_factorial2(N, F) :-
    N #> 0, N1 #= N - 1,
    n_factorial2(N1, F1),
    F #= N * F1.

When we call n_factorial2(N, 120), after finding the first solution N = 5, the program fall into a infinite search.

Constraint logic programming, same as its host logic programming languages, normally face the problem of divergence, which means it can lead to an infinite search if programmed without care.

At first, I am a little confused by the behavior of the non-termination. After ask for the help of clp(fd)’s author Markus Triska, various help I get from ##prolog irc channel and books, thesis on relation programming in miniKanren, I finally found out why.

The problem is although that system like clp(fd) and miniKanren is designed with strong termination guarantees, they’re still using the principle that goal appear first will also excute first, which means instead of doing the constraint clause F #= N * F1, it will now first excute n_factorial2(N1, F1) which finally diverges because there are infinite Ns which satisfy the constrains.

The short law that one program in logic/relational programming and constraint logic programming is that as stated in William E. Bryd’s disseration “unification should always come before recursive call, or calls to other serious relations”. For constraint logic programming, it is “constraint should always come before recursive call”.

Use constraint logic programming to solve fibonacci problem

This paradigm can be used to solve a wide range of relational programming on finite domain such as integers. So it is definitively can be applied to the fibonacci number problem.

Here is the fibonacci number problem in prolog.

n_fib1(0, 1).
n_fib1(1, 1).
n_fib1(N, F) :-
    N > 1, N1 is N - 1, N2 is N - 2,
    n_fib1(N1, F1),
    n_fib1(N2, F2),
    F is F1 + F2.

Sadly, translate the above program to clp(fd) does not produce a convergent program.

:- use_module(library(clpfd)).

% non termination due to infinite solution to constrain of F #= F1 + F2.
n_fib1(0, 1).
n_fib1(1, 1).
n_fib1(N, F) :-
    N #> 1, N1 #= N - 1, N2 #= N - 2,
    F #= F1 + F2,
    n_fib1(N1, F1), n_fib1(N2, F2).

The problem is clearly discussed in a excellent stackoverflow answer, in short, the problem is caused by the fact constrain F #= F1 + F2 has infinite solution, which will yield divergence. So the natural thought is that to add constrain F #> 0 to the program. But this still does not work, since the constraint F #= F1 + F2, F #> 0 still has unlimited solution.

The right way is as said in the stackoverflow answer, add the constriants F1 #> 0, F2 #> 0, be aware also to follow the law to do relational/logic programming and constraint programming, make them before the recursive call. So the final work solution is as follow:

:- use_module(library(clpfd)).

% terminate add constrains of F1 #> 0, F2 #> 0.
n_fib4(0, 1).
n_fib4(1, 1).
n_fib4(N, F) :-
    N #> 1, N1 #= N - 1, N2 #= N - 2,
    F #= F1 + F2, F1 #> 0, F2 #> 0,
    n_fib4(N1, F1), n_fib4(N2, F2).
    
?- n_fib4(N, 8).
N = 5 ;
false.

?- n_fib4(N, 13).
N = 6 ;
false.

?- n_fib4(N, 15).
false.

But the above problem has efficiency problem since naive fibonacci algorithm runs in exponent time complexity. This can be showed from the running time in prolog.

?- time(n_fib4(N, 55)).
% 196,914 inferences, 0.036 CPU in 0.036 seconds (100% CPU, 5402819 Lips)
N = 9 ;
% 6,662,749 inferences, 1.319 CPU in 1.463 seconds (90% CPU, 5051567 Lips)
false.

We can see when call n_fib4(N, 55) it will yield the first answer in around 0.04s, and will finish search for other answer in around 1.3s. If this is the case, then it will definitively not be able to solve large problems.

Use same idea in functional programming to accelerate n_fib4

In functional programming, we can add another argument to make it a dynamic like algorithm for fibonacci number, so that it can run in linear time as the following code in haskell

fib n = go n 0 1
    where
    go n a b | n==0 = a
             | otherwise = go (n-1) b (a+b)

We can use the same idea in constraint logic programming here.

:- use_module(library(clpfd)).

% accerlate n_fib4
n_fib6(0, 1, 0).
n_fib6(1, 1, 1).
n_fib6(N, F, F1) :-
    N #> 1, N1 #= N - 1, 
    F #= F1 + F2,
    F1 #> 0, F2 #> 0,
    n_fib6(N1, F1, F2).

Now we can call the following example and it’s much much faster than the naive recursive version.

?- time(n_fib6(N, 377, F2)).
% 77,602 inferences, 0.014 CPU in 0.014 seconds (100% CPU, 5468345 Lips)
N = 13,
F2 = 233 ;
% 868 inferences, 0.000 CPU in 0.000 seconds (99% CPU, 2376832 Lips)
false.

?- time(n_fib6(N, 378, F2)).
% 80,399 inferences, 0.015 CPU in 0.015 seconds (100% CPU, 5325256 Lips)
false.

Conclusion

Constraint programming is a really undervalued programming paradigm which is ignored by most people. But we should also notice that there is no free meal either, we should be very careful to add additional constrains as in the above example showed, but it is still a very very powerful paradigm for general relational programming which deserve to be known. It will extremely broaden one’s view of programming in general. You could find the whole experiment program from miscellaneous-code/fact-and-fib-in-clpfd.

Minimal object oriented in functional programming with a bunch of lines

2015-11-27T00:00:00+00:00

When I skimmed through the questions on Quora, there are plenty of questions about the comparison of object-oriented programming and functional programming. See resources section for details. And a lot of programmers with different background seems do not understand that object-oriented and higher-order function are closely related.

This post is just an attempt to show the internal relation of object-oriented and functional programming. Though a step by step code abstraction, how can we implement the core concepts in OO within 50 lines of code in a functional languages(Racket here).

Object-oriented programming 101

Since the population of object-oriented programmings of the last decade, I think most programmer will more or less know some OO concepts. Here I just list some basic concept of OO for a quick review.

In an purely OO languages(like Ruby and Smalltalk), everything is in terms as follow:

All values(the result of evaluating expression) are references to objects
Given an object, code “communiates with it” by calling its method(message passing).
Each object has its own private state. Only the object’s methods can directly access or update this state(Encapsulation).
Every object is an instance of a class
An object’s class determins the object’s behavior. The class contains method definitions that dictate how an object handles method calls it receives.
Inheritance is maily for code reuse and independent extension of the system.

So the definitions of classes are just like contracts of the different categories of objects, when the system is running, there are only objects, no classes at all. We can send objects messages, and the objects may change its internal states thought method calls.

Functional programming 101

The definitions of functional programming vary from first class function (even C++11) to purely functional languages like Haskell.

Since this is not a post talking about functional programming, we only use the less-strick version, which means:

Function as first class citizen, which means functions can be passed as argument, be returned as result and can be stored(Sometimes refered as higher-order function). They are just as normal variables in imperative languages
Referential transparency, which means give a function the same argument it will return the same result(Pure function).

This post will show how can one only use higher-order functions and mutalbe state to implement the core concept of OO in Scheme/Racket.

Step by step examples to implement OO concepts

OO with explict send interface

The first step is to use higher order functions as methods in an object and store the variables of object in a mutable association lists as the following code.

#lang racket

(struct object (vars methods))

(define (assoc-m name xs)
  (cond [(null? xs) #f]
        [(equal? name (mcar (car xs))) (car xs)]
        [else (assoc-m name (cdr xs))]))

(define (get obj field)
  (let ([ptr (assoc-m field (object-vars obj))])
    (if (false? ptr)
        (error "access non exsiting field:" field)
        (mcdr ptr))))

(define (set obj field new-val)
  (let ([ptr (assoc-m field (object-vars obj))])
    (if (false? ptr)
        (error "set non exsiting field:" field)
        (set-mcdr! ptr new-val))))

(define (send obj msg . args)
  (let ([ptr (assoc msg (object-methods obj))])
    (if (false? ptr)
        (error "method not found:" msg)
        ((cdr ptr) obj args))))

;; make one point object
(define p1
  (object (list (mcons 'x 0)
                (mcons 'y 0))
          (list (cons 'get-x (lambda (self args) (get self 'x)))
                (cons 'get-y (lambda (self args) (get self 'y)))
                (cons 'set-x (lambda (self args) (set self 'x (car args))))
                (cons 'set-y (lambda (self args) (set self 'y (car args))))
                (cons 'dist-to-origin
                      (lambda (self args)
                        (let ([x (send self 'get-x args)]
                              [y (send self 'get-y args)])
                          (sqrt (+ (* x x) (* y y)))))))))

In the above code, every object is a structure with two fields, vars is to use to store the variables of a object and methods is used to store methods of the object. The vars is a list of mutable cons cells since we need to update the value of object, and methods is a list of cons cell, because objects’ functions will not change overtime.

Then we define helper function assoc-m which when gived a field name, will return the mutable cons cell in the list. The get and set functions are just like get and set method in a OO languages, herein they are just helper function for every objects.

Finally we can define a send function which send the object a message. We predefined an object p1 for test purpose. We can use p1 like as follows

> (send p1 'set-x 3)
> (send p1 'set-y 4)
> (send p1 'get-x)
3
> (send p1 'dist-to-origin)
5

Transform to more OO like style using closure and dispatch

The above code actually isn’t like an OO languages because the weird send syntax, and normal OO language usually use obj.method style of code to do message call. We can transform the above code to “more OO like” using closure and dispatch.

For readers who are not familiar with closure, I will explain a lit bit about closure here. In short, closure is just a pair which store a) the function definition and b) the substitution environment when the function is defined. It is used to implement lexical scope, which maybe is one of the most important inventions in programming languages.

With lexical scope, we can verify the correctness of program by reading the program instead of debuging it and see the variable after each step. For a detailed discussion of closure, I recommend to read the book Let over lambda.

We can change the p1 definition to the following style.

(define ptx
  (let ([obj (object
              (list (mcons 'x 0)
                    (mcons 'y 0))
              (list (cons 'get-x (lambda (self args) (get self 'x)))
                    (cons 'get-y (lambda (self args) (get self 'y)))
                    (cons 'set-x (lambda (self args) (set self 'x (car args))))
                    (cons 'set-y (lambda (self args) (set self 'y (car args))))
                    (cons 'dist-to-origin
                          (lambda (self args)
                            (let ([x (send self 'get-x args)]
                                  [y (send self 'get-y args)])
                              (sqrt (+ (* x x) (* y y))))))))])
    (lambda (method . args)
      (cond [(equal? method 'get-x) (send obj 'get-x args)]
            [(equal? method 'get-y) (send obj 'get-y args)]
            [(equal? method 'set-x) (send obj 'set-x args)]
            [(equal? method 'set-y) (send obj 'set-y args)]
            [(equal? method 'dist-to-origin) (send obj 'dist-to-origin args)]
            [else (error "undefined method for object ~a" method)]))))

We can see from the above code, now the object is stored in a let binding, and we make ptx a function and dispatch different methods to the proper send operation on the obj object.

Then we can now call the object using the following syntax.

> (ptx 'get-x)
0
> (ptx 'set-x 3)
> (ptx 'set-y 4)
> (ptx 'get-x)
3
> (ptx 'dist-to-origin)
5

Add initialization function

The above code still have one serious problems. If I want to produce many objects, we should copy and paste the body of ptx many times. It is totally a disaster, since we are lazy and believe the DRY(don’t repeat yourself) principle.

We can abstract the creation of object as something like initialize method in ruby or __init__ in python. The following are the code.

(define (new-point [x 0] [y 0])
  (let ([obj (object
              (list (mcons 'x x)
                    (mcons 'y y))
              (list (cons 'get-x (lambda (self args) (get self 'x)))
                    (cons 'get-y (lambda (self args) (get self 'y)))
                    (cons 'set-x (lambda (self args) (set self 'x (car args))))
                    (cons 'set-y (lambda (self args) (set self 'y (car args))))
                    (cons 'dist-to-origin
                          (lambda (self args)
                            (let ([x (send self 'get-x args)]
                                  [y (send self 'get-y args)])
                              (sqrt (+ (* x x) (* y y))))))))])
    (lambda (method . args)
      (cond [(equal? method 'get-x) (send obj 'get-x args)]
            [(equal? method 'get-y) (send obj 'get-y args)]
            [(equal? method 'set-x) (send obj 'set-x args)]
            [(equal? method 'set-y) (send obj 'set-y args)]
            [(equal? method 'dist-to-origin) (send obj 'dist-to-origin args)]
            [else (error "undefined method for object:" method)]))))

Now we can use new-point to create as many as points as I like. It is the same as new objects in Ruby or Python.

> (define p1 (new-point 3 4))
> (define p2 (new-point 5 12))
> (p1 'get-x)
3
> (p2 'get-x)
5
> (p1 'dist-to-origin)
5
> (p2 'dist-to-origin)
13

A simple attempt to implement inheritance

We still lack one important concept in object-oriented programming: inheritance. A simple approach to handle this is just to add another filed in the object structure, which indicate the inheritance relation of the current object. If the object does not inherit from anything, just give it a default value false.

We need to modify the send function accordingly, since now if we can not find the proper method to execute, it may be contained in the ancestors’ objects, we need to find them on the chains to the root. The changed code is here.

(struct object (vars methods father))

(define (send obj msg args)
  (let ([ptr (assoc msg (object-methods obj))])
    (if ptr
        ((cdr ptr) obj args)
        (apply (object-father obj) msg args))))

(define (new-point [x 0] [y 0])
  (let ([obj (object
              (list (mcons 'x x)
                    (mcons 'y y))
              (list (cons 'get-x (lambda (self args) (get self 'x)))
                    (cons 'get-y (lambda (self args) (get self 'y)))
                    (cons 'set-x (lambda (self args) (set self 'x (car args))))
                    (cons 'set-y (lambda (self args) (set self 'y (car args))))
                    (cons 'dist-to-origin
                          (lambda (self args)
                            (let ([x (send self 'get-x args)]
                                  [y (send self 'get-y args)])
                              (sqrt (+ (* x x) (* y y)))))))
              #f)])
    (lambda (method . args)
      (cond [(equal? method 'get-x) (send obj 'get-x args)]
            [(equal? method 'get-y) (send obj 'get-y args)]
            [(equal? method 'set-x) (send obj 'set-x args)]
            [(equal? method 'set-y) (send obj 'set-y args)]
            [(equal? method 'dist-to-origin) (send obj 'dist-to-origin args)]
            [else (error "undefined method for object" method)]))))

(define (new-point-3d [x 0] [y 0] [z 0])
  (let ([obj (object
              (list (mcons 'z z))
              (list (cons 'get-z (lambda (self args) (get self 'z)))
                    (cons 'set-z (lambda (self args) (set self 'z (car args))))
                    (cons 'dist-to-origin
                          (lambda (self args)
                            (let ([x (send self 'get-x args)]
                                  [y (send self 'get-y args)]
                                  [z (send self 'get-z args)])
                              (sqrt (+ (* x x) (* y y) (* z z)))))))
              ;; father object of this object
              (new-point x y))])
    (lambda (method . args)
      (cond [(equal? method 'get-x) (send obj 'get-x args)]
            [(equal? method 'get-y) (send obj 'get-y args)]
            [(equal? method 'get-z) (send obj 'get-z args)]
            [(equal? method 'set-x) (send obj 'set-x args)]
            [(equal? method 'set-y) (send obj 'set-y args)]
            [(equal? method 'set-z) (send obj 'set-z args)]
            [(equal? method 'dist-to-origin) (send obj 'dist-to-origin args)]
            [else (error "unknow method call" method)]))))

Now we can create two init function, one for point and another for point-3d. The point-3d inherit from the point class. We can see it from the code. And the dist-to-origin procedure is something like overwritten method in OO languages.

> (define p3d (new-point-3d 2 2 2))
> (p3d 'get-x)
2
> (p3d 'get-y)
2
> (p3d 'dist-to-origin)
3.4641016151377544
> (sqrt (+ 4 4 4))
3.4641016151377544

Now the only thing we lack is some syntax sugar to change the domain specific OO language to a more familiar OO style like(Ruby code):

# encocing: utf-8

class Point

  def initialize(x, y)
    @x = x
    @y = y
  end

  def get_x
    @x
  end

  def get_y
    @y
  end

  def set_x(x)
    @x = x
  end

  def set_y(y)
    @y = y
  end

  def dist_to_origin
    Math.sqrt(@x * @x + @y * @y)
  end
end


ptx = Point.new(3, 4)
puts ptx.dist_to_origin

But it is not so important since they’re just syntax differences on the surface.

Conclusion

With the above step by step transformation, we can draw the conclusion that with the power of higher-order function and closure, we can easily implement core concepts of OO in functional programming languages with support for mutable state. OO is just a natural extension of functional programming with higher-order function and explicit states. The essential code is less than 50 lines(excluding the new init function). You can find the complete code with comments from repository m00nlight/oo-in-fp.

Resources

White Noise or not, does the stock prices have long term memory

2015-11-02T00:00:00+00:00

Traditional efficient market hypothesis(EMH) assume the fluctuations of stock market accord with normal distribution, which is widely know as a wrong assumption. In this post, I will show you the stock prices not only do not fit into the beautiful bell curve, but also has long-term memory.

Some backgrounds

What is efficient market hypothesis(EMH)

Efficient market hypothesis assume that the market fluctuations are random, thus no one can outperform the overall through selection of different stocks. But we always see there are some people who can constantly beat the overall performance while most of others do fairly worse most of the time.

And accroding to this theory, huge fluctuations should be extremely rare in stock market(due to the nature of Gaussian distribution), but we frequently see extremely huge fluctuations all the time(far more often than we expect).

Normal distribution

Why EMH not work

Many people attribute this to the fact that the stock market isn’t so efficient, since there is problems like information asymmetry, news leak etc. But it still can not explain so many crisis and bubble of mature market like US stock market.

From my point of view, the problem of EMH is that stock market system, like many other systems in the nature, are actually complex system. We can not only use purely mathematica formula or differential equations to model this system.

As a form of complex system, stock market contains a lot of characteristics commonlly seen in other complex system. Like:

Cascading failures(when stock prices crisis, more people sell their stock, which cause even steeper crisis)
Non-linear relationship
Hierarchical organization(there are banker, fund company, individuals investors etc.)
Emergent phenomena
Relationships contain feedback loops
…

The following picture shows the quntile plot of n days’ accmulat earning of Apple and Microsoft stock prices. I choose the two since the former represent a stock with roaring growth in the last few years, and the other is quite stable during the last decade.

Apple Quantitle plot of accumulate earning from 1 to 30 days

Microsoft Quantitle plot of accumulate earning from 1 to 30 days

The better the plot fit to the dashed line, the more random walk(or white noise) it is. As we expect, microsoft has better fit than Apple, especially when n is large(since it is more stable). But it is still by no means indicate that the fluctuations is a normal distribution. See the two ends of the plot, they drift off the dashed lines a lot, which indicate big fluctuations(bubble or crisises) occur more more often than what a normal distribution has.

So what?

Even if we accept that stock fluctuations do not follow the normal distribution, what does it actually meaning? Can we have better model to describe the stock market?

The answer is yes. We have some techniques to show that the stock market does actually have some type of long-term memory, which is called Hurst Exponent.

Hurse Exponent Definition

In short, Hurst Exponent is just a fractal dimension for time series data. It is a meansure of long-term memory of the time series.

The strick definition of Hurst Exponent is as following:

$E[{R(n) \over S(n)}] = C \times n^H ~~~~ n\to\infty$

where $E[x]$ is expect value, $R(n)$ is the range of the first n value, , $S(n)$ is the standard deviation of the first n values and n is the observation span.

The meaning of $H$ in different ranges are as follow:

$0.0 < H < 0.5$ means the sequence has long-term switching between high and low value, which means a high value will normally have low value as next and vice versa.
$H = 0.5$ means the time series is uncorrelate, or more techniquely, a random walk process
$0.5 < H < 1$ means the series have positive correlation, which means a high value will probabily follow another high value and vice versa

Steps to estimate the Hurst Exponent

Like most parameters in fractal, Hurst Exponent can not be simply calculated. We normally use some other techniques to estimate the value like box counting in fractal dimension estimation.

Because Hurst Exponent is some type of power law, there are normally two methods to estimate the parameter, one is to use log-log plot of the data, and do a linear model fit to evaluate the exponent, another is use maximum-likelihood fashion. The former is know to produce biased power law exponent estimation, but since it is much simpler, I will just use it in this post.

The basic idea is to use different range span n to calculate the value of $R(n)$ and $S(n)$ , and calculate the log of the expect value, do the same of ${n \over 2}, {n \over 4}, \ldots, {n \over 2^m}$ for the same. And plot the point in a two dimensional plan, and do a linear model fit to the data, the slope of the line will be the Hurst Exponent.

Here is the pesudo code:

def hurst_expoent(xs):
    def hurst_r(xs):
        m = mean(xs)
        # if xs = [x0, x1, x2, .., xn-1] then accumulate(xs) = [x0, x0 + x1, x0 + x1 + x2, ...]
        ys = accumulate(map(lambda x: x - m, xs)) 
        return max(ys) - min(ys)

    def hurst_s(xs):
        return std(xs)

    def hurst_rs(xs):
        if abs(hurst_s(xs)) <= 1e-6:
            return 1.0
        else:
            return hurst_r(xs) / hurst_s(xs)

    n = len(xs)

    line_points = []
    while n >= 2:
        xxs = partition(xs, n)  # partition the list into equal part of length n
        vs = map(lambda x: hurst_rs(x), xxs)
        line_points.append((log(mean(vs)), log(n)))  # base 2 log
        n = n // 2

    linear_model_fit(line_points)

I build an interactive demo in Mathematica to explore the result. You can try an online version by clicking here.

Apple's 1 day Hurst Exponent

Apple's 14 days Hurst Exponent

IBM's 14 days Hurst Exponent

Microsoft hurst exponent value from 1 day to 30 days' return

And we can see that the larger the n, the bigger the Hurst Exponent will be, which indicate more long-term memory of the data.

Conclusion

From all the above discussion, we not only showed that the fluctuations of stock market does not follow normal distribution but also it has some type of long-term memory. I always believe the stock market is something tightly related with complex system, which can not described by some simple equations. The market is not simply the sum of every individuals in the market, and should be studied from a totally different point of view.

I think to view it as a complex system is a good start point, and in my point of view, fractal and complex system are two of the most important discoveries of the 20th century. They have radically changed the way we see and understand our world and can use to explain a lot of phenomena we could not only use math of physics previously.

Believe it or not, non-linear, fraction dimension and chaos are main roles in our daily life. We should embrace new thoughts from complex system to help us better understanding the world.

Raspberry pi 2 with Mathematica inside

2015-11-01T00:00:00+00:00

A few days ago, I ordered a raspberry pi 2 ultimate starter kit set from amazon. This Sat. it was delivered to me and I immediately start to make it as a small computer for my usage.

You can see from the following picture, the little black box right of my keyboard is the Raspberry PI 2 computer.

Raspberry PI 2

The assembly process is quite easy, since I just plan to do basic things at the beginning. So the process is just to plugin in the 8GB sd card to the slot, put the motherboard in the box properly, and all is done. It is now a little cute computer.

Then we can just plug in keyboard, mouse, wifi module(which is include in the set), HDMI video cabel. The Raspberry PI will start when power on. At first, we need to install the OS like following:

Installation of Operating System

After installation, it will reboot to start. The first time will require you to do some configuration like configure locale, login options(GUI or terminal) etc. You can do whatever configuration for your tastes. After that, we can login to the system.

The main reason I bought this is that it included a pilot version of Mathematica with Raspberry PI in it. As a big fan of Mathematica, it is amazingly cheep to buy a device under $90 which contains a Mathematica(For comparison, the home edition of Mathematica I bought cost me $295).

So let us see the Raspberry PI 2 with Mathematica. :)

Starting Mathematica

Mathematica with Plot

Raspberry PI Store

Conclusion

I think Raspberry PI 2 is an excellent device for youth to explore the field of programming, which is also the target of Raspberry PI 2. The containing of Mathematica make it excellent quality and reasonable price device since I think Mathematica will teach you a lot of things other language can not. (Like higher order programming, functional programming, pattern matching etc, which are normally ignored in traditional programming education).

But since its cpu is based on the ARM architecture, and the performance of the cpu and memory is quite low, some program will not work properly on it. So I think it will not work as a fully functional computer for programmer like me. But it is still very valueable since it contains Mathemaitca, python programming languages and a lot of working example of game in it.

Is God a complexity system specialist?

2015-10-14T00:00:00+00:00

Look at the shcoking beautiful snow fakes images as bellow, are you amazed by how can the nature produce such things?

Snowflakes(Wiki)

Or maybe look at amazing plants as follow, can you think how God produce this from scratch?

Broccoli(Wiki)

Sunflower(Wiki)

But the answer may be simpler than you think. Because all these are some form of complexity system.

Complexity system in a netshell

First of all, the complexity here is not the same as algorithm complexity in computer science. It normally accompanies with some form of emergence phenomena.

Emergence means interactions of interest are non-linear, which yields levels of organization and hierarchies. – John Henny Holland

If we pay attention our life, we can find a lot of emergence phenomena around us. Like the flocking behaviour of birds, schooling of fish etc. Consider the following periodic table as an example, have you ever wonder why there are so many materials with only very few basic elements?

Periodic(Wiki)

Further,think about the carbon-based molecules like graphite and diamond, with the same basic element, only with a different structure, the diamond is one of the hardest materials on the earth while the graphite is one of the softest one.

Diamond vs Graphite(Wiki)

Many scientists have done a lot of research of such things, and showed that emergence can be actually achieved by leting the individuals follow very simple rules. So in simple terms, emergence phenomena means the whole is not only the simple summation of every individuals.

Behaviours of complexity system

Complexity system normally have several kind of behaviour in common. Here are just four normally observed behaviours in complexity system.

Self-organization : Ants, birds, fireflies etc are some famous example of this kind of behaviour. The system as a whole can show unbelievable intelligence with every individuals follow very very simple rules.
Chaotic behaviour : Means the system is initial conditions sensentive. The most popular term is butterfly effect.
Fat-tailed behaviour : Stock market bubbles and crashes as an example, we assume the market is efficient, but actually, the market’s fluctations seldom fit the beautiful bell-curve normal distribution. I will write a seperate post about this topic in the future.
Adaptive interaction : Agents in the system modify their strategies in diverse way.

We will not explain every behaviour here in details, since it will be out of the scope of this article. But just to give some high level ideas of complexity system.

Why are they choosed?

Take fractals as an example, consider the Koch Curve for partitular, we can make induction of the length of the curve as the following formula:

$L_n = \frac{4}{3} * L_{n - 1} = (\frac{4}{3})^{n - 1} * L_1$

So we can make induction to show that as n tends to infinity, the length of the curve aslo tends to infinity, although all the curve is in a bounded area. This is a extremely efficient way to organize complexity structure to make use of the space. Hence we can find enormously example of self similar or fractal structure in our daily life, coach line, surface of mountain, tree branch, broccoli, lung lobe and more and more, you name it.

Koch Curve(Wiki)

All these observation inspire me to look at problems in a new perspective. Just think about human neuros, they may be also follow the principle of emergence, every neuro play with very simple rules, but hundreds billion of neuros can produce extremely phenomena which we still can not understand.

Some other clues which indicate that human brain actually does not work in an artificial neural network style include that human brain is very energy saving, although brain is actually the most expensive organ in our body, it only cost about 400-500 calories. For comparison, Google Brain project, which is said to mimic the brain of human, but actually can only do very basic things of human like image classification, use 16,000 computers, and consume about 600KW. Can you image an brain cost so much? If every brain cost so much power, emission reduction should just be a joke, since everyone will cost much much more power than a car :).

Conclusion

I am always amazed by how clever and ingenious God create the world. He must be a extremely good complexity system expertise.

We have a long way to go on the way to the achievement of artificial intelligence. It is a hard way, which need the combination of a lot of different disciplines like biology, physics, math, complexity system, computer science and even philosophy. I don’t believe just to make machine learning more and more accurate itself is enough to lead us through this difficult path.

Why can't the existing machine learning approach bring to us real AI

2015-05-16T00:00:00+00:00

The last week, I spent a lot of spare time on doing program challenge problem on hackerrank, mainly focus on the artificial intelligence (machine learning) domain. Although the process was quite interesting and rewarding, it also strengthened my doubt about whether the current machine learning approach will bring us true AI in the future.

Here are some reason why I am not optimistic about nowadays’ machine learning approach.

Feature engineering is crucial in machine learning.

Coming up with features is difficult, time-consuming, requires expert knowledge. “Applied machine learning” is basically feature engineering. – Andrew Ng

Feature engineering use the domain knowledge of the data by the programmer, it is actually human intelligence, not artificial intelligence.

In some problems on hackerrank, some features would yield quite good result while some other features may produce very poor result. And combine features sometimes will reduce accuracy of learning algorithm. So you can not rely on improving the algorithm performance by thinking more and more features.

Parameter tuning

Machine learning algorithms normally come up with a lot of parameters, and these parameter can affect the performance of the learning algorithm quite a lot(up to %2 with my experience on the hackerrank problems).

Although parameter tuning can be done using some automatic methods, it actually requires some insight of the data by our human to find the right tuning direction. And parameter tuning is time consuming, and the parameters tunned on one dataset maybe produce poor result on another dataset, which is true most of the time.

Different learning algorithms have large gaps in result

Even with the same features, different machine learning algorithm can produce results which have huge difference. On one problem, only change the classification algorithm will improve the accuracy around 20%.

Although we can use some automated method to do the algorithm selection of learning process, it is still some kind of human intelligence instead of artificial intelligence.

More data normally beats better algorithm

From this blog post, the professor Anand Rajaraman showed some good example that by adding more independent data usually beats fine-tuning algorithm.

In fact, high quality data is another key to successful learning algorithm. Although we want to develop learning algorithm can learn pattern from the data, but normally more data will speak for themselves.

Will deep learning be a major breakthrough in AI?

Nowadays, deep learning is extremely hot. A lot of people seemed quite optimistic about that deep learning will lead us to true artificial intelligence.

Although I think in some sense, deep learning’s distributed representation and automatic feature learning are more useful than the traditional statistic learning approach. I still suspect whether it can bring us true AI in the near future.

Though I believe that human’s memory is in some distributed form, but I don’t think it is of the style deep learning represent it. I don’t believe that the human neuron store the real value of something, and use some so complex algorithm like gradient descent to learn the representation of something.

Conclusion

Essential, all models are wrong, but some are useful – George Box

I think we’re still very far away from true artificial intelligence. Although with more and more data and new methods like deep learning we can do a lot of things more useful using the machine learning approach, it is actually not true intelligence.

We have already experienced two fanaticism period of artificial intelligence , one in 1960s and another 1980s. Now it is like the on going third one. But we should clearly recognize that we still know very little about what is intelligence of human and where it came from. Sometimes, the bigger the expectation, the bigger disappointment.

I think machine learning can still do a lot of useful things for human. Just do not get your hopes too high that machine learning can produce true AI in the near future. The world that robots rule the world will still in the film longer than we think. :)

To bring us true artificial intelligence may require breakthroughs in different fields like neuroscience, philosophy, computer science etc. Even though someone may argue that the airplane is not like a bird, but it can fly, I want to remind them that the aerodynamics which make bird and airplane flying is the same, and we human have already extensive knowledge about that. So it should be the same for artificial intelligence, we can not expected too much when we actually know very little about the knowledge behind intelligence.

Hackerrank nCr

2015-04-11T00:00:00+00:00

nCr

Problem

The problem can be found here. The statement of this problem is extremely simple, in short, you need to calculate the ${n \choose r} \bmod M$ where M is 142857.

Lucas Theorem

If 142857 is a prime number, then we can just use the Lucas theorem to solve it. However, $142857 = 3^3 \times 11 \times 13 \times 37$ , that is why the problem is marked as an expert level problem in hackerrank.

The core idea of using Lucas theorem to calculate ${n \choose r} \bmod P$ where P is a prime number is to represent $n!$ as $a \times P^e$ where a is relative prime to P. To calculate the representation of $n!$ , we can group $1 \times 2 \times \cdots \times n$ of P elements each, so it can be expressed as $(1 \times 2 \cdots \times (P -1)) \times P \times ((P+1)$ $\times$ $(P+2) \cdots \times (2P - 1)) \times$ $2P \cdots$ , so that $1 \times 2 \times \cdots \times (P - 1)$ is relative prime to P, and each group mod P is of the same result.

And then use Wilson theorem which said that $(P - 1)! \equiv -1 (\bmod P)$ . We can finally get the solution to calculate $n! = a \times P^e$ . The python code as follow:

def fact_mod(n, p, facts):
    """
    Type :: (Int, Int, [Int]) -> (Int, Int)
    Suppose n! = a * p^e (mod p), then the function return (a mod p, e)
    facts is i!(mod p) for 0 <= i < p, use Lucas theory

    >>> facts = gen_fact_mod_prime(7)
    >>> fact_mod(5, 7, facts)
    (1, 0)

    >>> fact_mod(15, 7, facts)
    (2, 2)
    """
    if (n == 0): return (1, 0)
    (a, e) = fact_mod(n // p, p, facts)
    e += n // p

    if (n // p % 2 != 0): return (a * (p - facts[n % p]) % p, e)
    return (a * facts[n % p] % p, e)

Notice the tricky in the last two line of the above code. Since $(P - 1)! = 1 (\bmod P)$ , so if the number of group is even, then the result is just facts[n % p] % p, otherwise is -facts[n % p] % p, which is equivelent to (p - facts[n % p]) % p.

And then we can express ${n \choose r} = {n! \over {r! \times (n - r)!}}$ , calculate $n! = a_1 \times P^{e_1}$ , $k! = a_2 \times P^{e_2}$ and $(n - k)! = a_3 \times P^{e_3}$ . Then if $e_1 > e_2 + e_3$ , ${n \choose k}$ must be zero mod P, otherwise, the result should be a1 * modinv (a2 * a3, P) where modinv is the modulo inverse function.

def comb_mod(n, k, p):
    """
    Type :: (Int, Int, Int) -> Int
    Return C(n, k) mod p, p is a prime number.

    >>> comb_mod(5, 3, 7)
    3

    >>> comb_mod(6, 2, 7)
    1
    """

    if n < 0 or k < 0 or n < k: return 0
    facts = gen_fact_mod_prime(p)
    a1, e1 = fact_mod(n, p, facts)
    a2, e2 = fact_mod(k, p, facts)
    a3, e3 = fact_mod(n - k, p, facts)
    if (e1 > e2 + e3):
        return 0
    else:
        return a1 * modinv(a2 * a3 % p, p) % p

Generalize to this Problem

We can use very similar idea to calculate $n! \bmod P^a$ for $a\ge1$ . Let $b = max \{i | n! \bmod P^i == 0, i \in \mathcal{N} \}$ , then we need to calculate largest divisors of $n!$ relative prime to $P^a$ .

This time we group $1 \times 2 \cdots \times n$ into group each has $P^a$ items. We first pre calculate the array facts[0..P^a], facts[i] is the largest divisors of $i!$ relative prime to $P^a$ mod $P^a$ . Then we can use the same strategy when calculate $n! \bmod P^a$ .

def comb_mod2(n, r, m, pa, facts1):
    """
    Type :: (Int, Int, Int) -> Int
    m is of form p^a, and n is very large
    """
    p, a = pa

    def n_fact_fact(n):
        if n is 0 or n is 1:
            return 1
        elif n < m:
            return facts1[n] * n_fact_fact(n // p) % m
        else:
            a = facts1[m - 1]
            b = facts1[n % m]
            c = n_fact_fact(n // p)
            # print 'n = %d a = %d b = %d c = %d' % (n, a, b, c)
            return pow(a, n // m, m) * b * c % m

    def get_power(n, p):
        ret = 0
        while n > 0:
            ret += n // p
            n //= p
        return ret
    b = get_power(n, p) - get_power(r, p) - get_power(n - r, p)

    if b >= a: return 0

    m1 = n_fact_fact(n)
    m2 = n_fact_fact(r)
    m3 = n_fact_fact(n - r)

    return (p ** b) * m1 * modinv_table[(m2, m)] * modinv_table[(m3, m)] % m

The above code is just to calculate ${n \choose r} \bmod P^a$ . The function n_fact_fact is just to calculate the largest divisor of $n!$ mod $P^a$ which is relative to $P^a$ . And get_power is to get the largest number of b such that ${n \choose r} \equiv 0 (\bmod P^b)$ .

Finally, we can use Chinese Remainder Theorem to solve the problem. Since $142857 = 3^3 \times 11 \times 13 \times 37$ , we can calculate ${n \choose r} \bmod 3^3$ , ${n \choose r} \bmod 11$ , ${n \choose r} \bmod 13$ and ${n \choose r} \bmod 37$ , and the modulo numbers are relative prime to each other, so we can use it to finally get the answer.

You can find the whole solution in C++ and Python here.

HackerRank Minimum Multiple

2015-04-11T00:00:00+00:00

HackerRank Minimum Multiple

Problem statment

The origin statment of the problem can be found here.

In short, this problem ask you to do two operations on an array, one is to query the least common multipler(lcm) of an range [l, r], and another is to update one value of the array.

Since the maximum length of the array is $5 \times 10^4$ , an naive $O(n^2)$ algorithm will definitively not acceptable. So we need to find more efficient way to do the query and update operations.

Segment Tree

The solution is to use a structure called segment tree to make the query and update operations run in $O(log{n})$ time complexity.

Segment tree is often used in scenario of efficient range query and update. The following is an example of segment tree.

In segment tree, all the leaf nodes store the information of the segment value, and the internal node store the result of a range [l, r].

The pseudo codes of query and update are:

function query(f, l, r, root):
	if l <= root.left and root.right <= r:
		return root.val
	else if r <= root.leftChild.right:
		return query(f, l, r, root.leftChild)
	else if l >= root.rightChild.left:
		return query(f, l, r, root.rightChild)
	else:
		return f(query(f, l, r, root.leftChild),
			query(f, l, r, root.rightChild))

function update(f, idx, newVal, root):
	if idx <= root.left and root.right <= idx:
		root.val = newVal
	else if idx <= root.leftChild.right:
		root.val = f(newVal, root.val)
		update(f, idx, newVal, root.leftChild)
	else:
		root.val = f(newVal, root.val)
		update(f, idx, newVal, root.rightChild)

At first glance, the running time of query is not explicit $O(log{n})$ . But you can find the proof from the stackoverflow answer, which shows that the running time of the query function is actually $O(log{n})$ .

Solution

Then the solution is trival. We use an segment tree to store the lcm information. The node cover range [l, r] is the lcm of the elements in array[l..r]. So both the update and query operator can be done in $O(log{n})$ .

The following is a least common multiple segment tree example.

The circle nodes are leaf nodes and store the numbers in the array, and the rectangle nodes store range lcm information as described in the brackets in the node.

Here is the haskell code of the segment tree structure

data SegTree =
    Node {
      val                   :: Integer
    , left, right           :: Int
    , leftChild, rightChild :: SegTree
    } |
    Leaf {
      val         :: Integer
    , left, right :: Int
    }

initSegTree :: Int -> SegTree
initSegTree n = aux 0 (n - 1)
    where aux l r
              | l == r = Leaf {val = -1, left = l, right = r}
              | otherwise =
                  let mid = (l + r) `div` 2
                  in Node { val = -1, left = l, right = r
                          , leftChild = aux l mid
                          , rightChild = aux (succ mid) r
                          }


query :: (Int, Int) -> SegTree -> Integer
query range@(l, r) root
    | r < left root = 1
    | l > right root = 1
    | l <= left root && right root <= r = val root
    | otherwise =
        lcm (query range (leftChild root)) (query range (rightChild root))


update :: Int -> Integer -> SegTree -> SegTree
update idx newVal root
    | left root <= idx && idx <= right root =
      case root of
        Leaf {} -> root {val = newVal }
        _ -> root {val = lcm newVal (val root),
                 leftChild = lChild, rightChild = rChild }
    | otherwise = root
    where
      lChild = update idx newVal $ leftChild root
      rChild = update idx newVal $ rightChild root

In fact, since haskell is an pure functional languages, it should be a little trival to do programming related with states in it. But since the character of programming contests problems. We do not need to use state manipulation in haskell to get the problem accept, we can just use purely functional style to do the query and update operators using fold in haskell.

The following code is related with solving the problem.

processQueries
  :: [(Char, Int, Int)] -> SegTree -> [Integer] -> [Integer]
processQueries [] _ acc = reverse acc
processQueries (('Q', l, r):q) root acc =
    let ans = (query (l, r) root) `mod` modulo
    in processQueries q root (ans : acc)
processQueries ((_, idx, value):q) root acc =
    let oldVal = query (idx, idx) root
        newVal = oldVal * (fromIntegral value)
        nroot = update idx newVal root
    in processQueries q nroot acc

solve :: Int -> [Integer] -> [(Char, Int, Int)] -> [Integer]
solve n arr queries = processQueries queries tree []
    where
      tree = foldl' (\ root (idx, v) -> update idx v root)
             (initSegTree n)
             (zip [0..] arr)

The only thing one should notice is that the depth limit of stack on HackerRank is quite small. So we need to avoid too deep recursive call, and use the strick version of foldl' call in the solve function, and also avoid explicit recursive call in processQueries, but to optimize the function as an tail recursive call.

The full code can be found here.

Hello JekyllBootstrap

2015-03-04T00:00:00+00:00

Read Jekyll Quick Start for quickly set up a blog on github using jekyllbootstrap.

JB made blogging simple

Jekyll Bootstrap made it much simple to build an modern style of blog in several minitues.

Add math support to your blog.

First, make sure you upgrade to the default markdown kramdown of github pages, then add the fowllowing to your jekyllbootstrap theme’s default.html page in _include/theme/YOUR_THEME directory

<script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>

Now you can use TeX syntax to type in math symbols in your blog. For example, the following snippe of code will be rendered in TeX

$$\sum_{n=1}^\infty 1/n^2 = \frac{\pi^2}{6} $$

$\sum_{n=1}^\infty 1/n^2 = \frac{\pi^2}{6}$