I would recommend to let it buffer for a while. The phone also has a setting in android to speed up performance in the Android Settings.

Where in the android settings can I do dat? My ph is a moto droid

I am not sure what version of Android there is, but in Kitkat there was a feature called Download booster if you can download the video and watch it locally. (This is what I was trying to remember, its not actually a media loader specifically sorry about that. )