Как распарсить json?
MrLincomins, а где вы видите массив в исходных данных? Именно массив, как вы к нему обращаетесь, а не ассоциативный массив?
приведённая в вопросе строка, а конкретно
не является валидной json-строкой и поэтому не может быть распарсена стандартными средствами
Решения вопроса 1
Immortal_pony @Immortal_pony Куратор тега PHP
В рамках бредопрограммирования я написал функцию, которая превращает твою строку в валидный json
function repairUnqotedJsonValues(string $json) : string < $json = str_replace(": ", ":", $json); $json = str_replace("< ", "", ">", $json); $quotes = 0; $fixMode = false; $repaired = []; foreach (mb_str_split($json) as $char) < $add = [$char]; if ($fixMode) < if ($char === ":") < array_push($add, "\""); >if ($char === "," || $char === ">") < array_unshift($add, "\""); >if ($char === "\"") < $fixMode = false; >> $repaired = array_merge($repaired, $add); if ($char === "\"") < $quotes++; >if ($quotes === 2) < $fixMode = true; $quotes = 0; >> return implode("", $repaired); >Далее, после обработки строки с её помощью можно уже пользоваться json_decode:
$weirdString = file_get_contents('application.json'); $json = repairUnqotedJsonValues($weirdString); $user = json_decode($json); $name = $user->; print $name;Ну а правильным вариантом будет, конечно, хранение в application.json валидного json’а, а не строки в непонятном формате.
Как распарсить json идущий друг за другом со вложенностью?
мне нужно каждый десереализовать в объект естественно.
вложенность может быть любая в теории для такой задачи.
нужно нечто в духе «раз уже один открывающий < встретили, значит, если таковой встретится - его игнорируем и дальше ищем >. Если нашли >, то проверяем, чтобы между первым < и >было равное кол-во < и >.»?
Регулярки из гугла все перепробовал, ломаются на вложенности, а свою написать мозгов пока что не хватает, да и зачем, я уверен, кто-то уже делал.
ломается
итд
Или готов выслушать методы проще и лучше! может не вижу очевидностей
- Вопрос задан более трёх лет назад
- 395 просмотров
3 комментария
Простой 3 комментария
How to parse a JSON File in PHP ?
In this article, we are going to parse the JSON file by displaying JSON data using PHP. PHP is a server-side scripting language used to process the data. JSON stands for JavaScript object notation. JSON data is written as name/value pairs.
Syntax:
Example: The JSON notation for student details is as follows.
Advantages:
- JSON does not use an end tag.
- JSON is a shorter format.
- JSON is quicker to read and write.
- JSON can use arrays.
Approach: Create a JSON file and save it as my_data.json. We have taken student data in the file. The contents are as follows.
Use file_get_contents() function to read JSON file into PHP. This function is used to read the file into PHP code.
Syntax:
- file_name is the name of the file and path is the location to be checked.
- Use json_decode()function to decode to JSON file into array to display it.
It is used to convert the JSON into an array.
Syntax:
- $json_object is the file object to be read.
PHP code: The following is the PHP code to parse JSON file.
PHP
// Read the JSON file
$json = file_get_contents ( ‘my_data.json’ );
// Decode the JSON file
$json_data = json_decode( $json ,true);
// Display data
print_r( $json_data );
Output:
Array ( [Student] => Array ( [0] => Array ( [Name] => Sravan [Roll] => 7058 [subject] => java ) [1] => Array ( [Name] => Jyothika [Roll] => 7059 [subject] => SAP ) ) )
Whether you’re preparing for your first job interview or aiming to upskill in this ever-evolving tech landscape, GeeksforGeeks Courses are your key to success. We provide top-quality content at affordable prices, all geared towards accelerating your growth in a time-bound manner. Join the millions we’ve already empowered, and we’re here to do the same for you. Don’t miss out — check it out now!
Last Updated : 26 May, 2021
Like Article
Saved searches
Use saved searches to filter your results more quickly
Cancel Create saved search
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.
Efficient, easy-to-use, and fast PHP JSON stream parser
License
halaxa/json-machine
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Switch branches/tags
Branches Tags
Could not load branches
Nothing to show
Could not load tags
Nothing to show
Name already in use
A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Cancel Create
- Local
- Codespaces
HTTPS GitHub CLI
Use Git or checkout with SVN using the web URL.
Work fast with our official CLI. Learn more about the CLI.
Sign In Required
Please sign in to use Codespaces.
Launching GitHub Desktop
If nothing happens, download GitHub Desktop and try again.
Launching GitHub Desktop
If nothing happens, download GitHub Desktop and try again.
Launching Xcode
If nothing happens, download Xcode and try again.
Launching Visual Studio Code
Your codespace will open once ready.
There was a problem preparing your codespace, please try again.
Latest commit
Git stats
Files
Failed to load latest commit information.
Latest commit message
Commit time
September 30, 2022 18:09
November 18, 2023 19:05
February 3, 2022 21:56
December 24, 2021 17:31
October 10, 2022 13:55
November 18, 2023 21:46
February 19, 2022 21:36
February 3, 2022 21:56
October 12, 2022 13:40
December 2, 2018 11:36
November 18, 2023 23:13
November 19, 2023 13:39
May 4, 2023 11:56
December 21, 2021 16:03
README.md
Very easy to use and memory efficient drop-in replacement for inefficient iteration of big JSON files or streams for PHP >=7.0. See TL;DR. No dependencies in production except optional ext-json . README in sync with the code
- TL;DR
- Introduction
- Parsing JSON documents
- Parsing a document
- Parsing a subtree
- Parsing nested values in arrays
- Parsing a single scalar value
- Parsing multiple subtrees
- What is JSON Pointer anyway?
- GuzzleHttp
- Symfony HttpClient
- Available decoders
- Catching malformed items
- Streams / files
- In-memory JSON strings
- «I’m still getting Allowed memory size . exhausted»
- «That didn’t help»
- «I am still out of luck»
- Non containerized
- Containerized
- $users = json_decode(file_get_contents('500MB-users.json')); // this usually takes few kB of memory no matter the file size + $users = Items::fromFile('500MB-users.json'); foreach ($users as $id => $user) < // just process $user as usual var_dump($user->name); >
Random access like $users[42] is not yet possible. Use above-mentioned foreach and find the item or use JSON Pointer.
Count the items via iterator_count($users) . Remember it will still have to internally iterate the whole thing to get the count and thus will take about the same time.
Requires ext-json if used out of the box. See Decoders.
JSON Machine is an efficient, easy-to-use and fast JSON stream/pull/incremental/lazy (whatever you name it) parser based on generators developed for unpredictably long JSON streams or documents. Main features are:
- Constant memory footprint for unpredictably large JSON documents.
- Ease of use. Just iterate JSON of any size with foreach . No events and callbacks.
- Efficient iteration on any subtree of the document, specified by JSON Pointer
- Speed. Performance critical code contains no unnecessary function calls, no regular expressions and uses native json_decode to decode JSON document items by default. See Decoders.
- Parses not only streams but any iterable that produces JSON chunks.
- Thoroughly tested. More than 200 tests and 1000 assertions.
Parsing JSON documents
Parsing a document
Let’s say that fruits.json contains this huge JSON document:
// fruits.json < "apple": < "color": "red" >, "pear": < "color": "yellow" > >
It can be parsed this way:
use \JsonMachine\Items; $fruits = Items::fromFile('fruits.json'); foreach ($fruits as $name => $data) < // 1st iteration: $name === "apple" and $data->color === "red" // 2nd iteration: $name === "pear" and $data->color === "yellow" >Parsing a json array instead of a json object follows the same logic. The key in a foreach will be a numeric index of an item.
If you prefer JSON Machine to return arrays instead of objects, use new ExtJsonDecoder(true) as a decoder.
use JsonMachine\JsonDecoder\ExtJsonDecoder; use JsonMachine\Items; $objects = Items::fromFile('path/to.json', ['decoder' => new ExtJsonDecoder(true)]);Parsing a subtree
If you want to iterate only results subtree in this fruits.json :
// fruits.json < "results": < "apple": < "color": "red" >, "pear": < "color": "yellow" > > >
use JSON Pointer /results as pointer option:
use \JsonMachine\Items; $fruits = Items::fromFile('fruits.json', ['pointer' => '/results']); foreach ($fruits as $name => $data) < // The same as above, which means: // 1st iteration: $name === "apple" and $data->color === "red" // 2nd iteration: $name === "pear" and $data->color === "yellow" >Note:
Value of results is not loaded into memory at once, but only one item in results at a time. It is always one item in memory at a time at the level/subtree you are currently iterating. Thus, the memory consumption is constant.
Parsing nested values in arrays
The JSON Pointer spec also allows to use a hyphen ( — ) instead of a specific array index. JSON Machine interprets it as a wildcard which matches any array index (not any object key). This enables you to iterate nested values in arrays without loading the whole item.
// fruitsArray.json < "results": [ < "name": "apple", "color": "red" >, < "name": "pear", "color": "yellow" > ] >
To iterate over all colors of the fruits, use the JSON Pointer «/results/-/color» .
use \JsonMachine\Items; $fruits = Items::fromFile('fruitsArray.json', ['pointer' => '/results/-/color']); foreach ($fruits as $key => $value) < // 1st iteration: $key == 'color'; $value == 'red'; $fruits->getMatchedJsonPointer() == '/results/-/color'; $fruits->getCurrentJsonPointer() == '/results/0/color'; // 2nd iteration: $key == 'color'; $value == 'yellow'; $fruits->getMatchedJsonPointer() == '/results/-/color'; $fruits->getCurrentJsonPointer() == '/results/1/color'; >Parsing a single scalar value
You can parse a single scalar value anywhere in the document the same way as a collection. Consider this example:
// fruits.json < "lastModified": "2012-12-12", "apple": < "color": "red" >, "pear": < "color": "yellow" >, // . gigabytes follow . >
Get the scalar value of lastModified key like this:
use \JsonMachine\Items; $fruits = Items::fromFile('fruits.json', ['pointer' => '/lastModified']); foreach ($fruits as $key => $value) < // 1st and final iteration: // $key === 'lastModified' // $value === '2012-12-12' >When parser finds the value and yields it to you, it stops parsing. So when a single scalar value is in the beginning of a gigabytes-sized file or stream, it just gets the value from the beginning in no time and with almost no memory consumed.
The obvious shortcut is:
use \JsonMachine\Items; $fruits = Items::fromFile('fruits.json', ['pointer' => '/lastModified']); $lastModified = iterator_to_array($fruits)['lastModified'];Single scalar value access supports array indices in JSON Pointer as well.
Parsing multiple subtrees
It is also possible to parse multiple subtrees using multiple JSON Pointers. Consider this example:
// fruits.json < "lastModified": "2012-12-12", "berries": [ < "name": "strawberry", // not a berry, but whatever . "color": "red" >, < "name": "raspberry", // the same . "color": "red" > ], "citruses": [ < "name": "orange", "color": "orange" >, < "name": "lime", "color": "green" > ] >
To iterate over all berries and citrus fruits, use the JSON pointers [«/berries», «/citrus»] . The order of pointers does not matter. The items will be iterated in the order of appearance in the document.
use \JsonMachine\Items; $fruits = Items::fromFile('fruits.json', [ 'pointer' => ['/berries', '/citruses'] ]); foreach ($fruits as $key => $value) < // 1st iteration: $value == ["name" => "strawberry", "color" => "red"]; $fruits->getCurrentJsonPointer() == '/berries'; // 2nd iteration: $value == ["name" => "raspberry", "color" => "red"]; $fruits->getCurrentJsonPointer() == '/berries'; // 3rd iteration: $value == ["name" => "orange", "color" => "orange"]; $fruits->getCurrentJsonPointer() == '/citruses'; // 4th iteration: $value == ["name" => "lime", "color" => "green"]; $fruits->getCurrentJsonPointer() == '/citruses'; >What is JSON Pointer anyway?
It’s a way of addressing one item in JSON document. See the JSON Pointer RFC 6901. It’s very handy, because sometimes the JSON structure goes deeper, and you want to iterate a subtree, not the main level. So you just specify the pointer to the JSON array or object (or even to a scalar value) you want to iterate and off you go. When the parser hits the collection you specified, iteration begins. You can pass it as pointer option in all Items::from* functions. If you specify a pointer to a non-existent position in the document, an exception is thrown. It can be used to access scalar values as well. JSON Pointer itself must be a valid JSON string. Literal comparison of reference tokens (the parts between slashes) is performed against the JSON document keys/member names.
JSON Pointer value Will iterate through (empty string — default) [«this», «array»] or will be iterated (main level) /result/items > /0/items [] (supports array indices) /results/-/status , ]> (a hyphen as an array index wildcard) / (gotcha! — a slash followed by an empty string, see the spec) /quotes\» Options may change how a JSON is parsed. Array of options is the second parameter of all Items::from* functions. Available options are:
- pointer — A JSON Pointer string that tells which part of the document you want to iterate.
- decoder — An instance of ItemDecoder interface.
- debug — true or false to enable or disable the debug mode. When the debug mode is enabled, data such as line, column and position in the document are available during parsing or in exceptions. Keeping debug disabled adds slight performance advantage.
Parsing streaming responses from a JSON API
A stream API response or any other JSON stream is parsed exactly the same way as file is. The only difference is, you use Items::fromStream($streamResource) for it, where $streamResource is the stream resource with the JSON document. The rest is the same as with parsing files. Here are some examples of popular http clients which support streaming responses:
Guzzle uses its own streams, but they can be converted back to PHP streams by calling \GuzzleHttp\Psr7\StreamWrapper::getResource() . Pass the result of this function to Items::fromStream function, and you’re set up. See working GuzzleHttp example.
A stream response of Symfony HttpClient works as iterator. And because JSON Machine is based on iterators, the integration with Symfony HttpClient is very simple. See HttpClient example.
Tracking the progress (with debug enabled)
Big documents may take a while to parse. Call Items::getPosition() in your foreach to get current count of the processed bytes from the beginning. Percentage is then easy to calculate as position / total * 100 . To find out the total size of your document in bytes you may want to check:
- strlen($document) if you parse a string
- filesize($file) if you parse a file
- Content-Length http header if you parse a http stream response
- . you get the point
If debug is disabled, getPosition() always returns 0 .
use JsonMachine\Items; $fileSize = filesize('fruits.json'); $fruits = Items::fromFile('fruits.json', ['debug' => true]); foreach ($fruits as $name => $data) < echo 'Progress: ' . intval($fruits->getPosition() / $fileSize * 100) . ' %'; >Items::from* functions also accept decoder option. It must be an instance of JsonMachine\JsonDecoder\ItemDecoder . If none is specified, ExtJsonDecoder is used by default. It requires ext-json PHP extension to be present, because it uses json_decode . When json_decode doesn’t do what you want, implement JsonMachine\JsonDecoder\ItemDecoder and make your own.
- ExtJsonDecoder — Default. Uses json_decode to decode keys and values. Constructor has the same parameters as json_decode .
- PassThruDecoder — Does no decoding. Both keys and values are produced as pure JSON strings. Useful when you want to parse a JSON item with something else directly in the foreach and don’t want to implement JsonMachine\JsonDecoder\ItemDecoder . Since 1.0.0 does not use json_decode .
use JsonMachine\JsonDecoder\PassThruDecoder; use JsonMachine\Items; $items = Items::fromFile('path/to.json', ['decoder' => new PassThruDecoder]);- ErrorWrappingDecoder — A decorator which wraps decoding errors inside DecodingError object thus enabling you to skip malformed items instead of dying on SyntaxError exception. Example:
use JsonMachine\Items; use JsonMachine\JsonDecoder\DecodingError; use JsonMachine\JsonDecoder\ErrorWrappingDecoder; use JsonMachine\JsonDecoder\ExtJsonDecoder; $items = Items::fromFile('path/to.json', ['decoder' => new ErrorWrappingDecoder(new ExtJsonDecoder())]); foreach ($items as $key => $item) < if ($key instanceof DecodingError || $item instanceof DecodingError) < // handle error of this malformed json item continue; > var_dump($key, $item); >Since 0.4.0 every exception extends JsonMachineException , so you can catch that to filter any error from JSON Machine library.
Skipping malformed items
If there’s an error anywhere in a json stream, SyntaxError exception is thrown. That’s very inconvenient, because if there is an error inside one json item you are unable to parse the rest of the document because of one malformed item. ErrorWrappingDecoder is a decoder decorator which can help you with that. Wrap a decoder with it, and all malformed items you are iterating will be given to you in the foreach via DecodingError . This way you can skip them and continue further with the document. See example in Available decoders. Syntax errors in the structure of a json stream between the iterated items will still throw SyntaxError exception though.
The time complexity is always O(n)
TL;DR: The memory complexity is O(2)
JSON Machine reads a stream (or a file) 1 JSON item at a time and generates corresponding 1 PHP item at a time. This is the most efficient way, because if you had say 10,000 users in JSON file and wanted to parse it using json_decode(file_get_contents(‘big.json’)) , you’d have the whole string in memory as well as all the 10,000 PHP structures. Following table shows the difference:
String items in memory at a time Decoded PHP items in memory at a time Total json_decode() 10000 10000 20000 Items::from*() 1 1 2 This means, that JSON Machine is constantly efficient for any size of processed JSON. 100 GB no problem.
In-memory JSON strings
TL;DR: The memory complexity is O(n+1)
There is also a method Items::fromString() . If you are forced to parse a big string, and the stream is not available, JSON Machine may be better than json_decode . The reason is that unlike json_decode , JSON Machine still traverses the JSON string one item at a time and doesn’t load all resulting PHP structures into memory at once.
Let’s continue with the example with 10,000 users. This time they are all in string in memory. When decoding that string with json_decode , 10,000 arrays (objects) is created in memory and then the result is returned. JSON Machine on the other hand creates single structure for each found item in the string and yields it back to you. When you process this item and iterate to the next one, another single structure is created. This is the same behaviour as with streams/files. Following table puts the concept into perspective:
String items in memory at a time Decoded PHP items in memory at a time Total json_decode() 10000 10000 20000 Items::fromString() 10000 1 10001 The reality is even better. Items::fromString consumes about 5x less memory than json_decode . The reason is that a PHP structure takes much more memory than its corresponding JSON representation.
«I’m still getting Allowed memory size . exhausted»
One of the reasons may be that the items you want to iterate over are in some sub-key such as «results» but you forgot to specify a JSON Pointer. See Parsing a subtree.
«That didn’t help»
The other reason may be, that one of the items you iterate is itself so huge it cannot be decoded at once. For example, you iterate over users and one of them has thousands of «friend» objects in it. Use PassThruDecoder which does not decode an item, get the json string of the user and parse it iteratively yourself using Items::fromString() .
use JsonMachine\Items; use JsonMachine\JsonDecoder\PassThruDecoder; $users = Items::fromFile('users.json', ['decoder' => new PassThruDecoder]); foreach ($users as $user) < foreach (Items::fromString($user, ['pointer' => "/friends"]) as $friend) < // process friends one by one > >«I am still out of luck»
It probably means that the JSON string $user itself or one of the friends are too big and do not fit in memory. However, you can try this approach recursively. Parse «/friends» with PassThruDecoder getting one $friend json string at a time and then parse that using Items::fromString() . If even that does not help, there’s probably no solution yet via JSON Machine. A feature is planned which will enable you to iterate any structure fully recursively and strings will be served as streams.
composer require halaxa/json-machine
Clone or download this repository and add the following to your bootstrap file:
spl_autoload_register(require '/path/to/json-machine/src/autoloader.php');
Clone this repository. This library supports two development approaches:
- non containerized (PHP and composer already installed on your machine)
- containerized (Docker on your machine)
Run composer run -l in the project dir to see available dev scripts. This way you can run some steps of the build process such as tests.
Install Docker and run make in the project dir on your host machine to see available dev tools/commands. You can run all the steps of the build process separately as well as the whole build process at once. Make basically runs composer dev scripts inside containers in the background.
make build : Runs complete build. The same command is run via GitHub Actions CI.
Do you like this library? Star it, share it, show it 🙂 Issues and pull requests are very welcome.