I think you’re confusing the concept of references vs. value types and ByVal vs. ByRef. Even though their names are a bit misleading, they are orthogonal issues.
ByVal in VB.NET means that a copy of the provided value will be sent to the function. For value types (Integer, Single, etc.) this will provide a shallow copy of the value. With larger types this can be inefficient. For reference types though (String, class instances) a copy of the reference is passed. Because a copy is passed in mutations to the parameter via = it won’t be visible to the calling function.
ByRef in VB.NET means that a reference to the original value will be sent to the function (1). It’s almost like the original value is being directly used within the function. Operations like = will affect the original value and be immediately visible in the calling function.
Socket is a reference type (read class) and hence passing it with ByVal is cheap. Even though it does perform a copy it’s a copy of the reference, not a copy of the instance.
(1) This is not 100% true though because VB.NET actually supports several kinds of ByRef at the callsite. For more details, see the blog entry The many cases of ByRef