I feel your pain with RayCasts haha. My game relies entirely on using a ton of them from the player movement commands, selecting objects, firing laser beams, etc.
It is a LOT of work to set up a complicated game relying on so many RayCasts instead of simple collisions and more traditional player movement controls.
For just ignoring one layer and being able to hit others is very simple though once you know how to do it. Once you have a ton of different layer interactions that’s when it becomes so tedious to make everything work perfectly.
Here’s a little example of how to do what you want if I’m understanding your question correctly.
This example might be a little excessive (meaning some parts may be unnecessary I’d have to test to know for sure) but it works, some of the stuff I’m doing with Layer Masks seems strange but this is what I learned about it so far and it works at least.
//start position of Rayshot
public Transform rayStartPosition;
//ray shot length
float maxRayLength = 50;
//variables to store the rayHit info
GameObject rayHitObject;
String rayStringName;
Vector rayVectorHitPosition;
//declare LayerMask variables for player
LayerMask playerLayerMask;
int playerLayerMaskInt;
void Start()
{
//this is what seems weird but it works
//The "8" is the "User Layer" when you
//click on the "Add Layer" option in Inspector
playerLayerMaskInt = ~(1<<8);
playerLayerMask = playerLayerMaskInt;
}
//casting the ray example
RaycastHit rayHit;
if (Physics.Raycast(rayStartPosition.position, rayStartPosition.forward, out rayHit, maxRayLength, playerLayerMask))
{
//get the gameObject
rayHitObject = rayHit.collider.gameObject;
//get the name of object
rayStringName = rayHitObject.name;
//get the position of the rayhit
rayVectorHitPosition = rayHit.collider.gameObject.transform.position;
}
You can add more layers to ignore by declaring them like this example and simply adding a & between each LayerMask variable so it ignores every layer you want. Example:
if (Physics.Raycast(rayStartPosition.position, rayStartPosition.forward, out rayHit, maxRayLength, playerLayerMask & ignoreRayLayerMask))
So if you do it like that it’s actually pretty simple for what you are trying to do, but it is strange to learn it, and setting up an entire game around raycasts is very tedious for something like a Diablo 2 style of gameplay where everything is controlled by mouse clicks, or mobile screen taps.