The reference manual is for reference, if you need to use a particular class or function it's useful for checking what all the functions a class has and what arguments they take and what variables it uses. It also contains example uses of the functions which I find very useful.

To start with you should read the user manual and the guides to get you into grips with understanding the interface. Then you should try out some tutorials, using the reference manual if something doesn't make any sense.

There's no reason to just read one over the others, use all of them for their designed purpose.