Yeah, that's basically what this tutoral does, except it's for comments instead of votes. But either op type would work:
https://developers.steem.io/tutorials-ruby/get_account_comments
If you look at the JavaScript version, they do get_state which is discouraged. While it's discouraged, it might be a more efficient API call at the moment. When hivemind goes live, these kinds of queries will probably be a lot more efficient.
At the moment, the best, non-descouraged-yet-still-efficient way is to use SteemSQL.
RE: Greenshift 011: Finding the Most Recent Vote