Beginner : What am I doing wrong assigning materials?

I am an absolute beginner with Unity and not even dreaming of making a game yet. However I have created this beautiful fantasy/medieval type of castle with a 3D software, which I decided to bring in Unity to bring it alive.
I just don’t understand why is it so difficult to assign all the beautiful materials I have collected. Do I really have to drag them from the bottom to the scene window for every piece I want to color with the same material? The process is really slow, clumsy and awkward. Most of the time the material won’t even stick or misses the intended piece and just spreads all over the place instead.
I must’ve missed something.
Also, I’ve been looking for an asset that would allow me to pick a material and just keep painting my castle parts with one click. Anyone?

Thank you for reading.

your mats should be assigned in the modeling software then brought in and textures also some software has a smooth transition others take work more information is needed to answer this question for you if you are using blender say so nobody here shuns others for using free software.