It is a really good and tough question as there are always ups and downs in the life of a consultant.
To be frank, traveling (especially getting up very early on a Monday morning!) is tough, but some say "you get used to it". ? Also the fact that you do not see your friends and your partner on a daily basis is a real challenge for many of us. And from my personal view, it is the overall challenge to balance all of these expectations (from your client, your project manager, your friends and you partner, and also your own expectations). And there are days on which you do not fulfill all expectations, which can be a setback for yourself.
But every setback is also a chance for me to grow, and this is one of the things I do like about my job. During my time at RB, I got the chance to work with some of the smartest people I've met so far, the chance to learn and grow day-by-day and to be able to deliver (together with the client and my team) results that can or will change the way our clients work in the future. And of course, after a rough project, it is also very rewarding to receive the appreciation and a "Thanks" by our client for the hours and work we spent to come up with good results.
And, besides that, I am really happy to say, that some of these "people" (my RB colleagues) became my "friends".
I hope I could give you a glance on my perspective and my best/worst things as a consultant.