Are you talking about the WAN / public IP address that is assigned to your connection by their DHCP server / system ? Or are you talking about a LAN / private IP address that is assigned to one of your devices on your home network by the DHCP server that is built into the router ?
in windows, in the bottom left in the search type 'cmd' and open the Command Prompt.
Once it opens, type 'ipconfig /all' and the router address should be the one listed next to Default Gateway.
FWIW, that only shows the LAN IPv4 and the LAN IPv6 link local stuff, not the WAN public address for the connection..