--- title: "A story of compact() and extract()" url: https://www.exakat.io/a-story-of-compact-and-extract/ date: 2024-01-03 modified: 2024-01-03 author: "dams" description: "A story of compact() and extract() Compact() and extract() are two sides of the same coin. They are also a good part of the PHP story, with their close cousins,..." categories: - "Code auditing" tags: - "compact" - "extract" - "modernisation" - "usage" image: https://www.exakat.io/wp-content/uploads/2024/01/archive.320-7.jpg word_count: 2523 --- # A story of compact() and extract() # A story of compact() and extract() [Compact](https://php.net/compact)() and [extract()](https://php.net/extract) are two sides of the same coin. They are also a good part of the PHP story, with their close cousins, the variable variables. Let's review the usage of compact() and extract() and see how they can make it into the future of PHP. ## From variables to array and back `compact()` takes a list of variable names, as strings, and provide an array where the keys are those variable names, and the corresponding values are the value of the variables. `extract()` does the reverse operation, and create variables from an array made of `name => value`pairs. As is illustrated, `compact()` reads the variable name, then the variable and its value, then produces an array. There is no impact on the local variables, which are still the same after that process. On the other hand, `extract()` affects the local context, by updating, by default, or creating variables. This has very different implications. ## Alternative syntax to code compact() and extract() `compact()` can be emulated with the famous PHP variables variables. The variables variables are easily spotted in the code by the double (or more) `$$` in the name. The first (inner) variable is used to provide dynamically the name of the second variable. Then, its value is retrieved. This illustrates well the important role of `compact()` and `extract()`. They are a bridge between the world of variables and the world of data. Data is stored in strings (at least, here), and it is manipulated via variables. The variable names are usually hardcoded in PHP syntax. With variables variables, `compact()` and `extract()`, it is possible to go from variables to arrays and back. ## Another alternative syntax to compact() and extract() Another alternative to code `compact()` and `extract()` is using functions signatures and parameters. They actually have very similar features, and a few key differences. It is possible to turn an array of values into a set of variables by calling another method with the spread operator `...`. In PHP 8.0 and more recent, the named parameters matches the name of the index with a parameter. Later, inside the method, these will be real variables. Here, the spread operator, coupled with the named parameters, are acting as the extract() function. It may be more obvious when calling the function `call_user_func_array()`. Note that this alternative allows for a simple way to add type checking and name checking: those are ingrained in the method signature. Forbidding superfluous parameters is also handled with a `Unknown named parameter` fatal error. An equivalent of the `compact()` function is the `get_defined_vars()`, which list the local variables. Depending on the local context and usage, it might be fill the same role. ## Usage of compact() and extract() Let's now look at the actual usage of `compact()` and `extract()`. Out of 3000+ open source projects, they are used in respectively in 403 and 390 projects each. Since these two functions are supposed to be the opposite of one another, one would expect to find them used in equal quantities. This is almost the case, but not quite. And in details, there are 257 projects which are using both functions, and the rest are only using one of them. Another aspect of their usage is the options usage, for `extract()`. By default, the function overwrites the local variables with the one in the incoming array: it is EXTR_OVERWRITE. Yet, there are several options to alter that behavior. - EXTR_SKIP : 148 projects use this configuration - EXTR_OVERWRITE: 57 projects use this explicit configuration - EXTR_PREFIX_SAME : 11 projects - EXTR_PREFIX_ALL : 9 projects - EXTR_PREFIX_INVALID : 3 projects - EXTR_IF_EXISTS : 3 projects - EXTR_PREFIX_IF_EXISTS : 1 projects `extract()` offers a wealth of different behaviors, but the default and most obvious one is the one that is the most used: overwrite the local variables with the incoming values. The overwrite may become a problem when the variables are replaced by uncontrolled values. This makes `extract()` a security liability, as it open the door to altering the behavior of the current code. Note that `compact()` is not affected by this problem. Let's now review `compact()` and `extract()` individually. ## Some functions require an array with multiple values `compact()` is useful when several values have to be provided or returned from another method. One classic approach is to put them in an array. This is the case with option parameters. For example, the native `session_start()` expects several parameters. When the parameters have to be collected from several sources, it is convenient to put them in well named variables, and then, `compact()`them at the last moment. There are many custom functions from PHP itself, and various frameworks and CMS, which work that way. The array structure reduces the number of arguments to one, and makes the parameters optional : they may be omitted. Note that this behavior is very similar to using a lot of parameters and named parameters. The spread operator actually handles the extract operation. ## Shepherding variables after a long method The `compact()` approach is also a solution to modernize legacy code. When a big method is creating a lot of local variables that are difficult to untangle one from another, it might be easier to collect the needed values at the end of the method, in an array, and keep the previous code untouched. ## Sending variables to properties `compact()` produces an array of values. In this modern age of PHP, an object would be another viable option for speed and memory usage. The closest option is to use the `stdClass` class, via the `(object)`cast. Given that variables and properties share similar naming constraints, it is an easy step, if not a more performant one. To go beyond `stdClass`, one would need a complete class, with a constructor. It does require more code, but it is ultimately just as simple to use as a cast operator. ## The case for extract() Let's now take a look at `extract()`. It is a different beast to handle, since it writes in the local context. `extract()` is used to expand a list of values into variables. As for that, it is the opposite of `compact()`. When reviewing actual usage of code in projects, the source looks like the following: the names of the variables should tell you enough to understand it. Three aspects of these codes are interesting to observe. - `extract()` is used inside a method - `extract()` most often uses the default options - `extract()` works on generic values ## extract() often used inside a method `extract()` is often used inside a method. This means that creating the variables is inside the local context, and, it has a limited impact on the global scope, unless with an explicit `global` call. ## extract() almost always with default options Then, `extract()` is most often used with only one argument. The second argument is an option, which configures the way the function reacts when an existing variable is found. Here are the options and their usage. - EXTR_OVERWRITE : 53 projects * - EXTR_SKIP: 143 projects - EXTR_PREFIX_SAME : 11 projects - EXTR_PREFIX_ALL : 9 projects - EXTR_IF_EXISTS : 3 projects - EXTR_PREFIX_IF_EXISTS : 1 project EXTR_OVERWRITE is the default value: it overwrites any existing value with the new one. It is also the most commonly used value. Even EXTR_SKIP, which skips existing values, is used in almost half of the projects. As a general rule, `extract()` is used to overwrite existing variables, or to complete a set of existing variables. One very interesting option, that is seldom used, is EXTR_IF_EXISTS: this option only overwrites existing variables. This means that no new variable is created. The function shall create the expected variables first, with a default value. Later, they will be updated with the incoming array, when available. ## extract() works on very generic values The other aspect is the name of the data containers that are used with `extract()`. They are always very generic. `$parameters`, `$data`, `$this->INI`... are extracted. Their shape is varied : properties, parameters, variables with returned values. But their name is rarely more precise. That goes along with the little amount of checks: the code has a high level of trust in the source. - `$_POST`, or its cousins `$_GET`, `$_REQUEST`, etc. is a vague souvenir of `register_global`. Basically, it dumps the incoming variables from an HTML form into the current context. This is very old way of coding, unsafe. Those are rare. - `$this->INI` is a case of configuration. The configurations values are stored in a `.ini` file (or `.yaml`, or `.toml`...). To make them more convenient to handle than the `$INI['configuration_name']` syntax, they are turned into variables. The most important is that the list of configuration variables in the `INI` file is never documented anywhere. It must be flexible and accept anything, as configurations change often in numbers and shapes. - `$data` and `$parameters` are often used in templates for a view. The controller collects all the needed data, which are transfered as one group to a view class. To keep the template view simple, the incoming parameters are extracted as a list of variables. Again, in that situation, the extraction has little knowledge of which variables are provided by the controller, or needed by the view. Both are managing them as they see fit, and it is beyond the control of `extract()`. - `$db_row` is a variation of the templating system above. A row of value is extracted from the database, and then, turned into a list of local variables. This time, the list of variables is known, as the list of column in the `SELECT` SQL command; yet it is often arbitrarily decided by the author of the SQL command, not the receiving end. ## Like a foreach() with list () We'll finish that review with a feature similar to `extract()` that sits with foreach(). It is possible to use `list()` (aka, `[]`) to turn the values into a set of variables. Later, this saves some array syntax in the loop. ## The case for unknown extracted variables The most common case of usage for `extract()` is actually coming from a split in the code. One actor has to define an arbitrary number of values, each with a name. This means configurations files, tabular datasets (think SQL rows, but anything spreadsheet style), templating systems. Then, that mass of values, where both the names of the values and the values are arbitrary, is passed to a central system to process it. This is, in the same order than previously, the configuration parser, the database connector or the view system. Finally, all those variables are made available in the code, for convenience. Such approach explains why `extract()` is set to overwrite the existing variables: the incoming values have precedence over the default values. It is more convenient to overwrite the variables than to check if they exists. The superfluous values are usually ignored, as the code has no usage of them. Security gets concerned with this practice, as some intruder might find a way to perturbate the context or update an important variable. The superfluous values would probably better be processed with the `EXTR\_IF\_EXISTS` option: it only overwrites the variables if they exist, and ignores entirely the non-existing ones. `No default value, no extract()` is the motto. ## Possible future updates of compact() and extract() `extract()` looks like a relic from the past, coming from register_global era. But it actually serves a specific need: when a large number of values must be manipulated by name, but also be entirely decided by a pieces of code that are not the one doing the extraction. Processing data from a form, reading long configurations, providing lots of data to views and extracting rows from databases are all valid usages. It is a classic feature of many PHP applications around. Yet, it is also possible to provide a few recommendations for the usage of `extract()` and make it safer. - Avoid using `extract()` outside a method. The global context is also the one holding all the global values, with the potential to impact any other part of the application. So, the safest is to keep `extract()` in a method, with a well controlled amount of variables in a local scope, and little chances of overflowing anywhere. - Update the default option to `EXTR\_IF\_EXISTS`, instead of `EXTR\_OVERWRITE`. This means that incoming variables must be inited before being updated via `extract()`. No default value, no `extract()`. This is similar to the variable initialisation anywhere else in PHP: give it a default value to initialize it, and later, change it. - Consider using a method signature instead of `extract()` to publish lots of variables. Modern PHP is able to use `...` three dots to call a method with a set of arguments in an array. The signature of the method then provides variables, with a type and a default value. This basically works as `extract()`, with extended safeties. - Consider migrating this array to an object. Casting, for a lack of a better word, the array into an object and its properties, would help keep the values under control. In particular, properties can have a default value, and a type, which `extract()` is missing. From there, the object can dispatch values easily to the rest of the application.