Level 102
You cannot respond with a model as such. You can send string that the other end can parse.
Perhaps explain why it needs to be sent through Http (and to where) and why you need it to send models?
Jun 28, 2022
11
Level 13
stream data for performance
I have a situation where a large amount of data needs to be fetched from the database and then sent through HTTP, and i am looking for a way to improve the performance of it.
I am considering streaming the data but,
- I don't know how to properly do that
- i don't know if it is meant for this kind of problem.
To give an example, i have 2 apps.
App 1
The App 1 will get the data from the database, and it will return a stream response.
Route::get('/stream', function(){
return response()->stream(function() {
Items::query()->chunk(200, function($items){
echo json_encode($items);
});
});
}, 200, [
'Content-Type' => 'application/json',
]);
App 2
$response = $app1->get('/stream');
i want to get the data as models and not string
- If i do
$response->getContent()it returns false - if i do
$response->streamedContent()it returns a string with all the data
Level 13
The idea is to improve performance so that is something i want to try. So there is no particular reason to send it through HTTP.
As i described, i have 2 apps and one of the apps needs data from the other. Having access to the same db is not an option. Also taking into consideration that the amount of data is large, i thought that maybe i could stream the data.
Also i can't parse the data. If i try to do json_decode() it returns null. Do you mean to parse it directly from the string ?
Level 102
@orest but why would the other app ever need all of the data? Normally you would set up an api that the secondary site could use.
For example https://packagist.org/packages/spatie/laravel-json-api-paginate
Level 13
i want also to try to minimise the number of requests. And with streaming this can be achieved.
Level 13
Is there a way to return json from the stream and to be able to do json_decode on the other end ?
Level 102
@orest What are you getting currently? You can try sending a json resource https://laravel.com/docs/9.x/eloquent-resources#writing-resources
Level 13
Currently i do echo json_encode($items) in App 1
But in App 2
- if i try to do
json_decode($response->streamedContent());i get null. It seems that thestringthat is returned fromApp 1is not a valid json
Level 102
@orest you said that this gave you the data? $response->streamedContent()?
Level 13
$response->streamedContent() returns the data as a string, but that string is not a valid json
json_decode($response->streamedContent()); returns null
Level 102
@orest can you check what error it gives? And store the string somewhere to inspect (like a log file). Perhaps just send 2-3 records while debugging
Level 13
the json-last-error is 4
The problem is that if i do echo within a foreach or inside chunk() then for some reason the json format is messed up.
for example if i do the following, The json format is messed up
foreach($items as $item)
{
echo json_encode($item);
}
But if i do the following ( echo without foreach ), it works.
echo json_encode(Item::first());
Please or to participate in this conversation.