Going to throw it out there, but why would you do all of this sorting a filtering in the backend? Using (I assume you are using it) SQL to sort data in this fashion will be slow, and if you're not careful could easily cause a grid-lock in your database if it's not handled properly.
Why not return your data in a structured JSON format, and let the client side code handle how it's displayed/filtered/sorted? It's super easy with something like AngularJS, and will have no performance impact on your server.
But to answer your question, in my eyes: If your project is small, the repository pattern is probably overkill. However as your project grows, and you start to test code, using the repository pattern keeps your code testable and logical. It doesn't matter whether or not you're going to swap out another data source, it just provides a clean way to access your data. Let your controllers handle the flow, and repositories handle the data.