I first heard of them a few months ago but couldn't quite figure out the purpose...
The main purpose is for hands on learning. There's lots to see, touch, and even hear.
This post from Purpose Driven Home offers several more great reasons but, to be honest, my favorite thing about them is that they're a great way to keep the kids busy during tough times of the day (like when I'm cooking dinner) other than TV.
For the bug bin I used a bin I already had with short sides so the kids can reach in. I bought a 10 lb bag of black beans for the local international food store and raided the Target dollar section for all the buggy goodness.