Not all roads in OSM have speed limits attached. You could of course get involved in the OSM project and add the speed limits in your area.
The speed limit sign is very small and appears on the right hand side of the display in the middle and difficult to see. You can test it out by setting a departure and destination and then from the tools menu pick simulate route. if the streets have a speed limit set you will see it on the simulated route.