Light affecting only specific objects

Is it possible to assign a light to specific objects? Or have a light that affects all objects by default but not others manually specified?

Yes, this is what Layers are for.

Assign your objects to a specific layer (use an existing layer, or create your own custom layer). Then go to the light's inspector pane and adjust the "Culling Mask" setting to have it light (or not light) your desired layers.

It is important to note this does not work with baked lightmaps (as of 2019) there has been a bug open on this since 5.6. Looks like the only way to deal with this situation is to switch to a third party baker or maybe some of upcoming render pipelines will take care of it.