This is an interesting question. You must know what you want to get from the doctoral study. Most programs are designed to prepare you for a career in academia. If you can find a good supervisor to follow, it will worth your time and efforts.
Yes, the ranking itself is not as important as what you do and with who. It also depends on where you want to go post-PhD.
I think schools in the top 30 in general and top 10 in the particular specialization should have relatively good placement and worth your time. But if you don't really care about placement, any school in the top 100 is good for you.