How to connect wired home phone network to Xfinity phone service?

I have voip phone service from xfinity. I currently have a cable connecting my xfinity modem to a single phone. My home is wired with a landline home phone network (4 wire telephone cable, rj11 connectors). Is it possible to connect this landline network to the phone service I receive from xfinity and thereby be able to connect phones at various locations on the landline phone network? How do I do it? Can I connect the phone network to a router switch (rj11 at one end, rj45 at the other)? What does the wiring look like?