yeah, at ~100mbit speed tests you almost certainly have a bottleneck somewhere. 100mbit ethernet adapter is a good place to check. Also verify the ethernet cable you’re using. I’ve found some “cat 5e UTP” were missing the “twisted pair” part and that prevented it from reaching gigabit link speeds. The next is any switch you may have in line between phone and router. Next is the router. If the router has 100mbit switch ports, you’re bottlenecked no matter what you plug into it.
This is what we call a “bad idea”.