One of the many ways to do this is to use a combination of asymmetric and symmetric encryption, on the server you need to know [each] client's public key, on client you need to know your servers public key then you can exchange an RSA key or some symmetric key (specially if performance is an issue).