Solution:
((ViewGroup)scrollChildLayout.getParent()).removeView(scrollChildLayout);
//scrollView.removeView(scrollChildLayout);
Use the child element to get a reference to the parent. Cast the parent to a ViewGroup so that you get access to the removeView method and use that.
Thanks to @Dongshengcn for the solution