The key problem here is that the desire to conserve horizontal space clashes with our perception of how the language is written.
It's a relatively common pattern to collapse columns & panes into thin "toolbars" on the side of the screen, as seen in Outlook 2007:
Thus, the most obvious solution for the problem with Kanban board would be to minimize the column into a bar with vertical text:
download bmml source – Wireframes created with Balsamiq Mockups
But it doesn't work well even with the text written in capital letters (you don't want to see the small letter version) because w aren't used to reading text in that direction. In fact, there isn't even a consensus on which way to rotate the text for book spines. Had it been an interface in Chinese, Japanese, or Korean, then vertical order of letters would have been fine as those languages are commonly written top-to-bottom. Thus, whatever method is employed it must have the text written horizontally.
I see 2 possible solutions: the first one is easy to implement and the second one is more elegant.
This one is actually similar to the one that the OP has already tried. The key difference is to have toggle buttons for each of the columns instead of showing just the ones that are collapsed. This way the user can see the complete list of all available columns & their status (visible or hidden), but there's still the limitation of the horizontal space with narrow screens or a large number of columns/bins.
download bmml source
A far more elegant way of collapsing columns is to let the neighboring column to "slide" over it covering just enough to show that some content is hidden. When mouse hovers over the collapse column, it expands temporarily exposing the content & letting the user interact with it. This approach not only allows users to easily see the status of each column but also reduces the time it takes to check the content of any collapsed one since expansion requires only hovering versus clicking. The toggle can be made with something like a lock icon (locked position vs floating position).