Where to plug in round cable

EdieN46 That depends on the model of cable box that's in use, which is why those installation guides EG shared can help as they provide information for each piece of equipment. I can definitely confirm that the modem needs a coax connection. I'll reference the X1 TV Equipment page to help identify which cable box you have, and if you have anything other than the Xi5 or Xi6, the box will also need a coax cable.
If that's the case and you're looking to connect both pieces of equipment in the same room, you'll need a coax splitter and two additional coax cables apart from the one you already have. In that set up, you'll have one cable from the wall to the splitter, and one cable each from the splitter to the cable box and modem.
Please let us know if that information helps!