I don't have a definite answer for you, but I'll offer a couple of comments.

You might be a bit confused about shaders. 'Particle shaders' are just shaders whose parameters are suitable for rendering particle effects; they're not some sort of fundamental feature of Unity, nor do they really have any bearing on the problem at hand. You can certainly make an object translucent or transparent without dealing with 'particle shaders', and most likely without dealing with shaders directly at all.

Now, whether making parts of your mesh transparent is an appropriate solution here, I really can't say, since I don't fully understand what problem you're trying to solve. If the objective is to remove these parts of the mesh permanently, then that's probably not what you want. If however you want to hide parts of the mesh dynamically, you could do this be changing the vertex colors (assuming you're using an appropriate shader, but I believe there are built-in shaders that will handle this). Note that if the mesh is 'smoothed' (that is, vertices are generally shared), triangles that cross the box's boundaries may end up being translucent (and gradated) rather than fully transparent.

As for moving vertices, if you didn't 'disconnect' the moved vertices by deleting triangles, you'd end up with (presumably large) triangles connecting the 'main' part of the mesh with the 'moved' part, which I doubt is what you want.

You can edit the mesh in whatever way you want via the Unity API, so if you wanted to clip the mesh to the box (either roughly by removing triangles, or exactly by clipping triangles), you certainly could. However, what you describe sounds like it would probably be better handled by a dedicated modeling application (unless you specifically want this to be an interactive, in-game process).

You might be able to use the shaders in this topic on the Unity forums.