Extending WiFi in Old Home

We live in a 145 year old home and our WiFi coverage is terrible. We have a somewhat centrally located office with an Xfinity Gateway with wifi capability. It covers the spaces we most need it, but we're left using our cellular data in about 3-4 rooms outside of the range. We've had no luck with the plug-in extenders.

We do have the ability to run ethernet cables from the Gateway to reach other areas of the house with poor coverage. If I'm reading correctly, I think this begins to move us towards a solution, but I get confused at this point. Do I need a router to connect to this ethernet cable? An access point?