Apparently the network is extremely unreliable. When I ping the machines it goes fast, the response time is as expected but I get packetloss! Packetloss on a LAN! Something fishy is going on...
I'm thinking of replacing the TP cable with a new one but it's strange it could have been damaged...