I ran some experiments to firm up my geometric intuition about ReLU networks.

Without looking the answer, what is your intuition about the size of the VC-dimension of ReLU networks as a function of a number of weights and layers?

Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks https://arxiv.org/abs/1703.02930