OECD-IMF WORKSHOP Real Estate Price Indexes Paris, 6-7 November 2006 - Managing hedonic housing price indexes: the French experience
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
OECD-IMF WORKSHOP Real Estate Price Indexes Paris, 6-7 November 2006 Paper 12 Managing hedonic housing price indexes: the French experience Christian Gouriéroux (CREST and University of Toronto) and Anne Laferrère (INSEE and CREST)
Managing Hedonic Housing Price Indexes: the French Experience Christian Gouriéroux∗, Anne Laferrère† September 2006 Abstract Despite their theoretical advantages, hedonic housing price indexes are not so commonly used by statistical agencies or real estate professionals. Many published indexes still rely on mean or median prices, or favor repeat sales methods, which require less data and technicality, but are less accurate and ro- bust. In France, as in other countries where housing sales have to be recorded in front of a notary, complete data sets on transaction prices and charac- teristics of dwellings are available. Such data have been centralized since 1994, and a regular computation of quarterly hedonic housing price indexes has been done since 1998. This paper describes the institutional setting of housing transactions in France, and the collaboration established between the notaries and the national statistical agency (INSEE). The notaries are respon- sible for data collection and regular computation, whereas the national agency takes scientific liability for the method. The detailed transaction information remain proprietary data, but desaggregated indexes are publicly and freely available. This organisation and role assignment have proven their efficiency and might be extended to countries with similar institutional setting. Keywords : Housing Price Index, Hedonic Method, Pricing System. 1 Introduction The theoretical advantage of hedonic methods for computing housing price indexes has long been acknowledged (see e.g. Case et al., 1991). Indeed, this is the only way to control for changes in the quality mix of dwellings, whose transaction prices are observed. ∗ CREST (Centre de Recherche en Economie et Statistique ) and University of Toronto. † INSEE (Institut National de la Statistique et des Etudes Economiques) and CREST (Centre de Recherche en Economie et Statistique ), email: anne.laferrere@insee.fr. 1
For instance, other indexes based on mean observed trading prices can be biased since the observed sales are not a representative sample of the set (portfolio, or ‘basket’) of dwellings that one wants to follow. An index based on the median transaction price is less sensitive to extreme observed values, but still subject to selectivity bias, as the quality of the properties evolves over time. The hedonic approach assumes a pricing model where a dwelling is represented by a limited number of observed characteristics, with their own prices, whose combination (its quality mix) makes the dwelling value. The pricing model is estimated from observed prices and characteristics of traded properties. Then the estimated model is used to follow over time the estimated value of a chosen basket of dwellings, even if some types of dwellings in the basket have not been traded at each date. Such an achievement comes at a cost, since both prices and characteristics of properties need to be observed and recorded. This rarely happens for actual transactions. Hence hedonic methods are often applied to valuations by chartered surveyors, or to quoted asked prices, rather than to observed transaction prices. In countries such as the US where residential mobility is high1 , some have turned to repeat sales methods. The repeat-sales index is computed by comparing the fetched prices of the same dwelling at two different points in time, and assuming that the quality mix stays exactly the same. However, besides the need of a high turnover, there is no means to be sure that the dwelling is identical (rehabilitation is not usually recorded, apartments can be divided or reunited2 ), and the selection bias is still present, as the set of traded dwellings (and those with multiple sales) can be a non-representative sample of the basket of interest. The high cost or even the impossibility of observing transaction prices and charac- teristics of the traded dwellings explains why a regular computation of hedonic indexes by statistical agencies or real estate professionals is not common3 . Many official indexes still rely on mean or median prices, or favor repeat sales methods which are less data demanding. In countries where the law requires housing sales to be recorded in front of a notary, data on transaction prices and characteristics of properties can be available. France is such a country, where the data on sales have been collected and centralized since 1994, and made possible the computation of quarterly hedonic housing price indexes since 1998. This paper describes the institutional setting of housing transactions in France, the main indexes,and their diffusion policy (Section 2), together with the way the data are collected, and the collaboration established between the notaries and the national statis- tical agency (INSEE). The notaries are responsible for data collection and computation, whereas the national agency takes the scientific liability for the hedonic method. The database is described in Section 3, and the hedonic specifications are presented in Section 4. A by-product of the hedonic method is a valuation expert system, briefly described in Section 5. This job organisation and role assignment for the notaries and National Statistical Institute have proven their efficiency, and might be extended to countries with similar institutional setting. 1 The annual residential mobility rate is about 17 to 18 percent in the US compared to 8 to 9 percent in France (Long, 1991; Baccaı̈ni, 2001). 2 This may explain why the method is employed for single family units, which are more easily identified by their address than apartments in a building. Some repeat-sales methods are combined with hedonic models for observed characteristics (see for instance Quigley, 1995, or Englund, Quigley, Redfearn, 1998) 3 Vrancken (2004) reports seven hedonic price indexes only, for second-hand housing, in Hong-Kong, Norway, Sweden, Switzerland and the UK, respectively. 2
2 The French institutional setting The French institutional setting is characterized by a network of notaries (notaires, in French) who have a monopoly in registering real estate transactions, and by a national statistical agency. In France, all real estate transactions have to be registered in front of a notary who has a monopoly. The role of a notary is to verify the existence of property rights, to draft the legal sale contract and deed, to send the records to the Mortgage Register (Conservation des Hypothèques)4 , and to collect the stamp duty for the government5 . A notary is both a public officer (officier ministériel ), and a private professional6 . Thanks to this feature of the French legislation7 , a notary has access to the transaction price, together with the dwelling characteristics that are written on the sale contract. Moreover, each notary has to send information on the price fetched by the property to the tax authorities, since the sale tax is function of the price. The corresponding data are appropriate for computing hedonic housing price indexes. They cover all sales, and thus there is no problem of sample representativeness (see the discussion below); they provide actual transaction prices and are not submitted to the uncertainty of a valuation process; the series are available over a long period with regular availability and continuity thanks to a stable legislation; the data frequency is adequate, as the notaries have to send the information and pay the tax to the Finance Ministry within 24 hours of a sale. The central statistical agency, that is the National Institute of Statistics and Economic Studies (INSEE8 ), is in charge of providing official statistics. Among other price data, it is responsible for the retail price indexes, the industrial price indexes, or the construction cost index. Up to the end of the 1990s, INSEE published no housing price index. The city of Paris was an exception, as a ‘Notaires-INSEE’ quarterly index was created in 1983 for second-hand apartments in Paris. INSEE helped defining segments and provided weights from the Population Census; then the index was computed by the notaries as a weighted average of transaction prices9 . In 1997, the Conseil Supérieur du Notariat (CSN), that is the National Union of Notaries, decided to create a price index for dwellings located outside the Paris region, the so-called Province. They turned to INSEE for advice. INSEE agreed to provide a methodology, because a public service of reliable housing price indexes was missing in France. To ensure long-term involvement of both parties, formal agreements were signed in 1998 and 1999 between the CSN and INSEE, and in 2000 and 2002 between the CINP and INSEE, for renovated hedonic indexes. 4 There are 354 decentralized property registers. 5 Or the Value Added Tax in case of a new construction. Notaries also collect the capital gain tax when applicable. 6 As a public officer his/her fees are regulated by law; they include a fixed part and one part roughly proportional to the sale value. 7 Notaries with similar duties exist in Belgium, The Netherlands, Morocco, etc. 8 Institut National de la Statistique et des Etudes Economiques. 9 More precisely, this index was computed by the Chambre Interdépartementale des Notaires Parisiens (CINP), that is the Parisian branch of the profession. The 72 segments were defined by crossing the number of rooms, the date of construction and the level of comfort. 3
2.1 A quarterly INSEE monitoring The notaries collect the data and compute the indexes at their own cost. By-products of the index computation are sold by the notaries to finance the data collection and the indexes updating. They go from part of the database, statistics on buyers and sellers, to a complete valuation system of dwellings and an expertise on real estate prices10 . INSEE does not compute the indexes but is answerable for the index method. For this purpose a quarterly quality control of the main indexes has been established. It relies on information on the data gathering (time of integration in the databases, quality controls) and on the comparison of the evolution of means prices and indexes for different regions and categories, in order to detect a possible structural modification. The volumes of sales, their structure by dwelling type (typically, the number of rooms) are followed and compared to the reference stock. Zones with extreme variations of price or volume compared to the preceding quarter or to other zones are also detected and checked for potential errors. 2.2 The published indexes Some 23 sub-indexes are currently published at the national level; they are publicly avail- able and free. Thirteen sub-indexes concern the apartments: Paris, the seven départements of the Petite and Grande Couronne of Paris11 , the towns of Lyon and Marseille, the urban units of more than 10,000 inhabitants (city centers, and suburbs), the small urban units and rural areas. Ten indexes are house sub-indexes, one for the Province, seven for the départements of Ile-de-France, and two for the Rhône-Alpes and the Provence-Alpes-Côte d’Azur regions. Various indexes at a larger geographical level are obtained by combining appropriately the sub-indexes; they concern the Petite Couronne, the Grande Couronne, the Ile-de-France, Rhône-Alpes and Provence-Alpes-Côte d’Azur regions, the Province, and France, both for houses and apartments, and for the two types of housing together. In some urban units or regions with enough sales, local indexes are also computed, but not yet all published by INSEE (see Fig.1). They will be published in a near future12 . All indexes can be found in the Bulletin Mensuel de Statistique (BMS), in January, April, July, and October, which is regular publication of INSEE, now entirely electronic, as well as on the INSEE website. Each published index is identified in the BMS by a code. They can also be found at //http/www.indices.insee.fr (Indices et séries statistiques, Con- struction Logement, Indices trimestriels des prix des logements anciens). In each case, in the first week of quarter t + 1, two indexes are provided quarterly that are a provisional index for quarter t − 1 and a revised final index for quarter t − 213 . For instance, on July 5 2006, the revised index for 2005 Q4 and the provisional index for 2006 Q1 are published. The main indexes are presented in Table 1. 10 The question of the cost is not dwelt upon here. Quarterly or annual press conferences held by the notaries of the Paris area are available at //http/www.paris.notaires.fr. For the rest of France see, http://www.immoprix.com/. 11 First outer ring and more remote suburbs, respectively. 12 The next region to have their own index are likely Nord-Pas-de-Calais, Pays de la Loire, and Midi- Pyrénées. 13 Base 100 of the indexes was fixed at the second quarter of 1994 for Paris, at the fourth quarter 1994 for the Province. 4
The institutional combination of a central statistical agency and a monopolistic net- work of notaries was at the root of the making of French hedonic housing price indexes. However, agreement on the need for housing price indexes was only a first step. There is a long way from the drafting of a sale contract to the publication of the index. 3 Database The drafting of a housing sale contract by notaries is not enough to make a reliable on line data base. The contracts are paper documents, sometimes heavy, and they are not written in a totally standardized way all over the country. To make a proper data base, the information has to be normalized and coded. The operation is costly, as a deed has many pages, and there are some 850,000 to 900,000 sales per year14 . Each of the 4,600 notaries is asked to send for key-boarding an extract or a photocopy of the sale deed, plus some extra notes on the dwelling characteristics15 . This is done on a voluntary basis. In the near future the sale contracts should be normalized and computerized, the process will use electronic mail and become much cheaper. This is not yet the case, even if the first tests for electronic contracts have been conducted in 2005. The data on a particular sale are integrated in the database within 2 to 3 months from the date of the signature. The speed at which each notary sends the data is crucial for the index quality. The best incentives to induce a notary to send his/her data are still being experimented. Before turning to email in the future, sending reminders twice a month by mail, including pre-filled and pre-paid envelopes seem to work best, along with additional phone calls. In the Province, the average time between a sale and the reception of the data is now 57 days; then, it takes 40 more days to integrate the transaction in the database. The delay is less in the Paris region. The index is restricted to arm-length transactions of second-hand dwellings16 . To enter into the index a dwelling has to be free for occupation (not rented at the date of the sale), only used for habitation (no professional use), and has to be acquired in full property by a private individual or by a SCI (Société civile immobilière17 ). Exceptional homes such as single rooms (service rooms), attics, artist studios, or castles are excluded. Those restrictions eliminate about 15 percent of transactions. There are two databases. The base BIEN, managed by the CINP, covers the Ile-de- France, that is Paris and the Paris region18 . The base Perval, managed by the Perval society for the CSN, covers the rest of France. There are 86 departements outside Ile- de-France19 . Together they included some 9,7 million transactions at the end of 2006, 30 percent in Ile-de-France, and 70 percent in the Province. This includes all real estate sales, 14 Among which some 90 percent are second hand dwellings which enter the scope of the index. 15 The systematic data collection necessary to make an hedonic index was decided at the end of the 1970 in Paris and in the 1990s for the rest of France. In 2004, about 20 people worked on the data collection, for the Province, and 15 for the Paris region. 16 Second-hand dwellings are distinguished from new dwellings from the way they pay taxes. New dwellings are submitted to value-added tax (VAT), which is lower than stamp duty. The first sale of a new building taking place 5 years after construction is no longer under the VAT regime, and enters into the index. 17 A family civil company for real estate investment. 18 The departements of Seine-Saint-Denis (93), Val-de-Marne (94), Val d’Oise (95), Essonne(91), Haut- de-Seine (92), Yvelines (78) and Seine-et-Marne(77). 19 Corsica and the French overseas territories are left out for the time being. 5
including for instance parking lots, new buildings, or land. As only second-hand houses and apartments are included in the hedonic index computation, 5,4 millions observations are used for the housing price indexes. Roughly half of them are apartments, half of them are houses. In 2006, some 780,000 new observations were added in the databases, among which 520,000 were second-hand sales of houses or apartments. The distinction between the Paris region and the Province is due to the history of centralization in France. As the Paris region, and the city of Paris itself, concentrate a large part of the wealth, the oldest historical database is the one collected by the CINP, as early as 1979 for Paris and Petite Couronne, since 1995 for the Grande Couronne. The database for the Province was created in 1990 and became operative in 1993. The making of the indexes brought the Parisian and Province databases closer. For instance a sale of a Parisian dwelling made by a notary of Province is now included in the Parisian database and vice-versa. 3.1 Coverage rate Since the data collection is made on a voluntary basis, the rate of coverage of the notary database is not 100 percent. For instance, 71 percent of the notaries of Province sent some data in 2006, whereas the rate is around 85 percent percent in Ile-de-France. The rate of coverage of the total housing transactions by the notary database is not perfectly known, because there are no other official statistics on housing transactions. The notaries from Ile-de-France collect statistics on their activity by a special survey. The survey does not separate housing from other real estate transactions. The overall coverage rate, computed by dividing the number of transactions in the database by the total number of transactions as measured by the survey on activity, was 85 percent in 2004 (89 percent for Paris, 88 percent for the first outer ring of Paris, and 78 percent for the more remote suburbs). An indirect way to estimate the coverage rate is to use the amount of stamp duties collected in each of the French départements, as known from the Tax authorities (Direction Générale des Impôts). Dividing the total tax by the tax rate (4.8 percent) provides the total sale value, for each department. By comparing it with the total sale value of the notary database at the same geographic level, one gets a coverage rate, in value (not in number of transactions)20 . The estimated average coverage rate in 2003 was 66 percent, that is, 83 percent in Ile-de-France and 64 percent for the rest of France. It varies from one place to the other. It was lower than 30 percent in 12 départements, between 30 percent and 50 percent in 23, between 50 percent and 70 percent in 36 départements and over 70 percent in 23 départements. Actually a 100 percent coverage rate is not necessary to compute a hedonic index. As seen below the method is based on the valuation of a fixed basket of properties, defined over 4 years of transactions. The structure of the basket is close to that of all dwellings, as known from the Population census. At least for observed characteristics, there is no 20 The method is not perfect as the tax rate is now the same for housing and other real estate trans- actions, which are no more separated in the tax statistics. If the coverage rate was the same for housing and other real estate, this feature would not be a problem. However, before 1999 when the tax rates were different for housing and other real estate, the coverage rate was higher for housing. Assuming that the differential between the coverage rate for housing and for other real estate transactions is constant over time at the département level, and that the share of housing among all transactions, as known from year 1999 (when the distinction was possible), is also constant, a coverage rate can be computed in each département (Friggit, 2003). 6
obvious bias in the properties that the notaries choose to send to the data base. Once the reference basket is fixed, the hedonic method is immune to selection bias, that is from the fact that the sales on given period are a non random sample of the stock of dwellings, and that registration in the database is also potentially non random (see below). It is however important to check that the coverage rate does not fall below a minimal level to insure that the transactions are sufficient for an accurate estimation at a particular local level. We turn back to this feature of the index below21 . 3.2 Characteristics of dwellings, and treatment of non-responses The database is anonymous to comply with the French law. The precise address of the dwelling is included but is not made public, and is not used in the index computation. The only location characteristic is a municipality code (code commune), close to a ZIP code, corresponding to a town or village (there are more than 36,000 communes in France), with an added neighborhood code (code quartier) when the commune is large enough22 . A neighborhood can for instance be one to four zones within an arrondissement in Paris. Dwellings are separated between houses and apartments. Note that the French housing park is divided nearly equally between houses and apartments. The way houses are constructed differs widely; brick dominates in the North and East, stone and concrete in the rest of France; constructions in wood are rare. A majority of houses are detached and located outside city centers in suburbs, or in villages, except in some regions were town houses can be found23 . The quality of apartments is linked to their date of construction. In all cases, the location tells much on the dwelling appearance and quality, not only in terms of neighborhood characteristics, but also in terms of building characteristics. For instance 19th century Hausmannian construction in Paris is of better quality than constructions of the same period in other areas. This is why hedonic regressions are estimated at a detailed local level, and why the models may include neighborhood dummy variables and cross effects (see below). Besides the zone and date of the sale, the observed dwelling characteristics are the following: surface (in square meters), time of construction (8 categories: < 1850, 1850- 1913, 1914-1947, 1948-1969, 1970-1980, 1981-1991, 1992-2000, > 2000), number of rooms (from 1, to 5 and more), number of bathrooms (0, 1, or 2 and more), number of garages or car parks (0, 1, or 2 and more) and for apartments, floor level ( 1st, 2nd, 3nd, 4th, etc...), presence of a lift, existence of a service room (0, 1 , 2 or more). For houses, the number of levels (1, 2, 3 or more), the presence of a basement and the surface of the plot are also known. The rate of non-response varies among the explanatory variables (Table 2). In case of non-response, either the sale does not enter into the index computation (for instance when the surface is unknown), or the characteristic is imputed from econometric models estimated on complete data (Table 3). 21 The coverage rate is also important to consider when the database is used to follow the activity of the housing market. Once the total amount of sale is known in each department (from tax data), dividing it by the average transaction price (from the notary database) provides an estimation of the number of sales. 22 Which side of the street, even or odd number, and even geocoding is also registered but not used in the hedonic computation. 23 According to the French Housing survey of 2001, 58.4 percent of houses are detached, 24.3 percent are semi-detached, and 17.3 percent are grouped. 7
4 The Hedonic Method The basic assumption common to all hedonic price indexes is that each dwelling is defined by the combination of a fixed number of characteristics, its quality mix, that enters the consumer’s utility. Among all hedonic housing price indexes that we are aware of the French index has the unique features of combining a large number of geographic zones/strata and the quarterly estimation of so-called ‘reference stocks’ of dwellings in each zone. This section describes those features in some details. Defining Zones/strata Dwellings (houses and apartments are separated all along) are assumed to be stratified into zones where prices are homogeneous and price evolutions are roughly parallel. It is important to estimate the hedonic models on homogeneous price zones, that are zones where prices are not too different, and move in the same way over time. Since the strata used for the publication of an index are not necessarily homogeneous, it is necessary to cut or group then. The homogenous segments have been defined locally by interviewing real estate experts. Then a tree analysis has been applied to aggregate similar segments. Ideally a model will be estimated per segment and the elementary geographic zones can represent rather small sub-markets. Practically we have been limited to a little less than 300 zones to ensure a sufficient number of sales per zone (over 400 per year). Typically, for large cities, above 10,000 inhabitants, a zone is a city center or a city suburb; a zone is a group of rural areas or smaller towns in less densely populated regions; it is close to an arrondissement in Paris. In a given zone, the price index is defined as the ratio of the estimated value of a reference stock of dwellings, a basket of houses, to its value at the base period of the index. For each quarter, the value of each dwelling in the reference basket is estimated from the prices of all observed sales by means of the hedonic econometric models that have been estimated on the sales of the ‘estimation period’. Reference stock The principle of the hedonic method is to correct for the variations of the structure of the sales at a particular date of observation. It is achieved by estimating the value of a fixed stock of dwellings at each date. The index follows the price of the dwellings in this reference stock. The reference stock is made of all sales during the period 1998-2001 in each of the 296 elementary zones/strata. It excludes sales in the extreme quantiles of the distribution of prices per square meter. The size of the reference stock in each zone is on average 2,800 dwellings, which represent about 1,220,000 dwellings for the whole stock (Table 4, col.4). This feature of the hedonic method makes it immune to selection bias.24 Hedonic pricing models Hedonic pricing models relate the prices (more precisely, the logarithm of the price per square meter) to the characteristics of the dwellings. The characteristics include the location (a neighborhood within a zone), and the quality of the dwelling itself. Each model is estimated on a stock of transactions called estimation stock. It includes all dwelling sales during the 1998-2001 period, except the transactions for which the number of rooms is not known, or the estimated price was found ex-post to differ from the observed price by more than two standard-errors. It is close, but not equal, to the reference stock defined above (see Table 4). The econometric estimations are made separately in each elementary 24 Contrary to a method based on including time dummies in hedonic regressions estimated on all recorded sales at each date. 8
geographical zone. The model is the following : 3 X 4 X K X Log pi = Log p0 + αa Ya,i + θt Tt,i + βk Xk,i + i (1) a=1 t=1 k=1 where pi denotes the price per m2 of dwelling i, Ya,i is a dummy variable for the year of sale of dwelling i, Tt,i a dummy for the quarter of sale of dwelling i and Xk,i , k = 1, . . . , K, are continuous or dummy variables computed from the dwelling characteristics. They can include nonlinear or interaction effects. For instance the presence of an elevator is crossed with floor level. The coefficients of the model characterize the prices of the characteristics levels, which together define a reference dwelling, the price of which is p0 25 . The variables X include the number of rooms, the floor, the number of levels, the av- erage size of rooms, the presence of a service room, a parking, a terrace, a balcony, a base- ment, or a garden, the number of bathrooms, the period of construction, the condition26 . Some estimated models include also a neighborhood dummy, and, in some specifications, the number of rooms is crossed with the neighborhood dummies. The lot size is included for houses. Remember that each model is estimated in a particular zone/strata, and thus all variables are de facto interacted with the zone. The choice of the explanatory variables including interactions has been done by an automatic classification and robustified, by a reduced rank analysis [see Gouriéroux, Jasiak (2006), chapter on multiple scores]. Two examples of hedonic models are reported in this paper. A first one for houses in the outskirts of Paris (Seine et Marne), a second one for houses in Dijon, a city of Burgundy (Tables 5 and 6). The dependent variable is the logarithm of the price per square meter (for apartment) or the total price (for houses) in Euros. The goodness of fit quality of the hedonic regressions as measured by the determination coefficient R2 , varies between 0.18 and 0.70 for apartments, and between 0.50 and 0.80 for houses. The number of observations ranges from 1,721 to 19,342. For individual cross-section data, values of R2 in the range of 0.25-0.40 for 1000 to 3000 observations and around 20 variables are considered good. This is what is obtained in most zones. Current value of the reference dwelling The same type of model is used at the current period τ , with the same reference dwelling of price p0,τ . The price per square meter of dwelling j sold in period τ is written as27 . K X Log (pj,τ ) = Log (p0,τ ) + βk,τ Xk,j,τ + j,τ . k=1 25 The reference dwelling is one of a precise quarter and year of sale. The value of a dwelling with the same characteristics, but sold at a different time is computed from p0 by multiplying by the corresponding quarter and year parameters exp θt and exp αt . 26 In Ile-de-France the variables ‘fair condition’ and ‘terrace or balcony’ are not known. 27 The evolution of the price of the reference dwelling is the core of the index construction. For this reason it must include seasonal and cycle effects. This is why the quarter and year parameters are not in the current period model, while they were introduced in the first model because the estimation was made over more than one quarter. The price for a dwelling of quarter (a, t) would be: Log (p0,a,t ) = Log p0 + αa + θt . 9
The period τ is chosen according to the type of index. More precisely, the index for a quarter t is computed over all arm-length transactions of a period τ ending with quarter t28 . Let us now explain how the price of the reference dwelling is computed from data on current sales. Let us assume that the βk,τ coefficients are known, and denote pej,τ the price that would fetch dwelling j with the characteristics of the reference dwelling, then: K X pj,τ ) = Log (pj,τ ) − Log (e βk,τ Xk,j,τ . k=1 p̃j,τ defines the ‘reference dwelling equivalent price’ of dwelling j, τ . The model can be rewritten as : Log (e pj,τ ) = Log (p0,τ ) + j,τ . Hence, if the βk,τ coefficients were known, the logarithm of the price of the reference dwelling Log (p0,τ ) would be estimated as the mean of all estimated prices: Jτ 1 X Log(b p0,τ ) = Log(e pj,τ ), Jτ j=1 where Jτ is the number of transactions at period τ . In practice, the hedonic models are found to be very stable over time, and it is assumed that the relationship between the characteristics and the price of a house is fixed, in a given zone, for a period of up to around five years29 . This allows to replace the βk,τ coefficients by the βbk estimated over the reference period. It simplifies the quarterly computation, of hedonic prices as they involve no further econometric estimation: K X pj,τ pj,τ ) ' Log (pj,τ ) − Log (e β̂k Xk,j,τ = Log [ PK ]. k=1 exp( k=1 β̂k Xk,j,τ ) Then, the log of the price per square meter of the reference dwelling in period τ , is estimated by a geometric mean of the ‘reference dwelling equivalent prices’ of the Jτ dwellings sold in period τ : Jτ Jτ 1 X 1 Y Log pb0,τ = Log pej,τ = Log( pej,τ ), Jτ j=1 Jτ j=1 28 Up to the end of 2003, the Parisian index was computed on a six-month basis, hence τ = [t − 1; t]; indexes for the Province were annual, τ = [t − 3; t]. From 2004 on they are all pure quarterly indexes, τ = t, which makes them more reactive and allows to study seasonal price variations. However, quarterly indexes at a more local level remain semestral or annual to ensure a sufficient number of transactions in the zone. Monthly indexes are currently tested for Paris. 29 P3 P4 The models assume that the time effect is captured by the term a=1 αa Ya,i + t=1 θt Tt,i and that the coefficients βbk are time invariant during the years following the estimation period. The time invariance assumption was checked. It was verified that the difference between the estimated value of dwellings with characteristic Xk and their actual sale price, that is the residual ui , satisfies the stochastic assumption of the model, and does not include an unobserved deterministic component. The time evolution of the mean of the residuals in some zones was computed for each of the coefficients βbk , k = 2 . . . , K. They were found stable over time. After a maximum of 5 years, they are checked and changed updated if necessary. This has been done in 2002-2003, with no major effect on the index profile. 10
or: Jτ ! J1 Y τ pb0,τ = pej,τ . j=1 Current value of the reference stock Once the value of the reference dwelling has been estimated, the estimated value of any dwelling of the reference stock can be computed, and, by aggregation, the value of the stock itself. The computations are made per zone. For this reason, let us re-introduce the index s of the zone. The value of dwelling i of the reference stock of zone s in the current period τ is estimated from its characteristics Xk,i,s , which are time invariant, by definition of the reference stock. The approached value is: K X pb∗ i,s,τ = exp(Log pb0,s,τ + β̂k,s Xk,i,s )Ai,s , k=1 where Ai,s is the surface of dwelling i, s. The estimated current value of the Ns dwellings of the reference stock of zone s is obtained by summation: Ns X W cs,τ = pb∗ i,s,τ . i=1 In the same way, the value of the reference stock is estimated at the base period of the index, denoted t = 0,. We get: Ns X K X W cs,0 = exp(Log pb0,s,0 + βbk,s Xk,i,s )Ai,s . i=1 k=1 Quarterly computation of the index The elementary index for zone s measures the evolution of the value of the reference stock of that zone s. It is given by: PNs b0,s,τ + K P b W i=1 exp(Log p k=1 βk,s Xk,i,s )Ai,s cs,τ It/0 (s) = = PNs PK b . W cs,0 exp(Log pb0,s,0 + i=1 βk,s Xk,i,s )Ai,s k=1 The index of zone s can also be written as: It/0 (s) = exp(Log pb0,s,τ − Log pb0,s ), and involves only the evolution of the price of the reference dwelling. The computation of the index at date t does not require the computation of the implicit value of each dwelling of the reference stock; the coefficients Log pb0,s,τ are obtained by: Jτ K 1 X X Log pb0,τ = Log (pj,s,τ ) − β̂k,s .X k,s,τ , Jτ j=1 k=1 11
where X k,s,τ is the mean of the Xk,j,τ variables for the Jτ transactions of the current period in zone s. Aggregate indexes Most elementary indexes per zone/stratum are not published. They are aggregated at larger geographical levels. For instance, the index for the ‘Province’ measures the evolution of the value of the whole reference stock of Province. This index can be written as P c W cτ Ws,τ It/0 = = Ps , W c0 s W cs,0 where the summation is made on the zones of the Province. It can be interpreted as a mean of the elementary indexes per zone, weighted by the total sales value in the zone in the reference stock: ! X W cs,0 It/0 = P c It/0 (s). Ws,0 s s Practically, the weights of some indexes are corrected by a parameter δs for zones where the notary database is deemed to be non exhaustive30 . The main Notaires-INSEE indexes are presented in Fig.2 (for apartments) and Fig.3 (for houses), together with their rate of increase. There are strong seasonal effects, espe- cially for houses, which may be linked to residential mobility of families and the school year calendar. INSEE also publishes seasonally adjusted housing and apartment price indexes. To summarize, the process involves several steps. The first four steps are done once and for all, and only updated every five years: step 1 Define zones (strata), where the price evolution is assumed to be homogeneous; step 2 Define a hedonic pricing model, that is introduce correction coefficients for quality effects, for each zone; step 3 Estimate the correction coefficients from an estimation stock of dwellings in each zone; step 4 Compute the value of a reference stock at the base date for each zone; The three following steps are repeated every quarter. step 5 Compute the value of the reference stock, from data on all current period sales per zone; step 6 Compute the price index as the evolution of the value of the reference stock between base and current date; step 7 Publish indexes and sub-indexes by aggregation of local zone indexes. 30 Corrected weights are fixed and estimated from stamp duty returns and correspond to the value of the reference stock in each zones. 12
Note that the quarterly computation of the index involves no econometrics. This feature makes it most attractive as the word hedonic is sometimes perceived by statistical agencies as synonymous of sophisticated and time consuming. 4.1 The model updating As the index is based on the valuation of a fixed basket of dwellings (the reference stock), the question arises of the updating of the basket. This is done every four or five years. We have to check for the stability of the models, their specification, and the local baskets themselves since the dwelling park is constantly evolving over time with new construction, destruction and rehabilitation. The zones/strata themselves may have to be redefined, or at least checked, as population is moving and being redistributed over the territory. The first revision took place in 2003. Some zones were redefined; the period of refer- ence, hence the basket, was updated (going from 1992-1996 or 1994-1996 to 1998-2001); the specifications were only marginally changed. This updating has no major effect on the indexes. 5 Expert System The construction of an hedonic price index is based on an econometric pricing model, which explains how the price of an appartment or house depends on its characteristics. The estimated model can be used to predict the price of any mix of characteristics, as done in the construction of the hedonic price index itself. By using the information on the estimated variance of the error term and the estimated variance of the beta coefficients, the model can also be used to get a 95 % prediction interval for any mix characteristics, that are a minimal and a maximal. This approach has been followed for building a pricing expert system, which is available on line, and is one of the source for financing the construction of the indexes. Such an expert system can be used for different purposes as a source of information on market prices before a transaction, or as a source for checking ex-post it a transaction price is compatible with the market, for instance to detect a possible fraud to tax payment. Such an expert system is also required for the implementation of the new regulations in Finance (Basel II), or Insurance (Solvency II). Indeed, the banks, credit institutions and insurance compagnies have a significant part of their portfolio directly invested in real estates, or indirectly since real estate is the standard collateral for mortgages or firm loans. In the current regulations the value of this portfolio has to be computed and updated very frequently. A pricing expert system is the natural tool for computing the values of real estate portfolios. 6 Conclusion Thanks to the conjunction of sales data, good will and accurate methodology, reliable housing price indexes now exist for France. These three elements are necessary and it is important that they persist in the long run. The data should go on being collected, that is the notaries have to settle on a durable way of funding them. The tax authorities are 13
unifying and computerizing the real estate sale documents. A side effect will likely be a reduced cost for data gathering and a better quality of data. But the information needed for the hedonic models, and not requested by tax authorities, has to be provided for the index and the hundreds of small notary practices have to be motivated. This leads to the second element, good will. It is fuelled by information about the use of the indexes. To the notaries, they should become a trademark, and the valuation system linked with the indexes should prove useful and a mean to make the enterprise profitable. On the INSEE and academic side, and for the general public, the mere existence of reliable indexes and of all the related informations, has begun to fuel new types of studies. As prices can be compared both in space and over time, they can be introduced in microeconomic models of agents decisions, and provide more reliable guidelines to public and individual choices. As housing and more generally real estate prices and consumer prices evolution can differ widely, it is of primary importance for economic policy to make use of both. As for methodology, its unique feature is the use of the valuation of reference parks at a detailed geographic level. Finally, the assumption of time stability of the model implies that there is no further econometric estimation in the given period of computation of the index, which saves time and cost. This feature of the hedonic method makes it attractive for government agencies. The first updating of the period base, the reference stock and the model specification just took place. It was rather easy to perform and without major effect on the index profile, which comforts the long term relevance of the hedonic methodology. 14
Reference Baccaı̈ni, B. 2001, Les migrations internes en France de 1990 à 1999: l’appel de l’Ouest, Économie et Statistique, 344, 39-79. Beauvois, M, David, A., Dubujet, F., Friggit, J., Gouriéroux, C. , Laferrère, A., Mas- sonnet, S. and E. Vrancken, 2006, Les indices de prix des logements anciens, version 2 des modèles hédoniques, INSEE Méthode, 111, 151 p.. Case, B., Pollakowski, H. and S. Wachter, 1991, On Choosing Among House Price Index Methodologies, Journal of the American Real Estate and Urban Economic Associ- ation, 19(3), 286-307. David, A., Dubujet, F., Gouriéroux, C. and A. Laferrère, 2002, Les indices de prix des logements anciens, INSEE Méthode, 98, 119 p.. Englund, P., Quigley, J., and C. Redfearn, 1998, Improved Price Indexes for Real Estate : Measuring the Course of Swedish Housing Prices, Journal of Urban Economics, 44, 171-196. Gouriéroux, C., and J. Jasiak, 2006, Econometrics of Individual Risks : Credit, Insur- ance and Marketing, Princeton University Press. Friggit, J. 2003, Taux de couverture des bases notariales, Note from the Conseil Général des Ponts et Chaussées, January 7. Long, L.H. 1991, Residential Mobility Differences Among Developped Countries, In- ternational Regional Science Review, 14, 133-147. Quigley, J. 1995, A Simple Hybrid Model for Estimating Real Estate Price Indexes, Journal of Housing Economics, 4, 1-12. Vrancken, E., 2004, Foreign house price indices, CINP (Chambre Interdépartementale des Notaires de Paris), working paper, Paris. 15
Table 1. The official indexes Web site Paper publication Type of index code code 081767865 00000 00 France 086909774 00000 10 France, apartments 086909673 00000 20 France, houses 086937763 14000 20 Ile-de-France 086910582 14000 10 Ile-de-France, apartments 086911592 13000 20 Ile-de-France, houses 086937662 15000 10 Ile-de-France (Paris excluded), apartments 067517858 11000 10 Paris, apartments 086909875 15010 10 Seine et Marne, apartments 086909976 15020 10 Yvelines, apartments 086910077 15030 10 Essonne, apartments 086910178 15040 10 Haut-de-Seine, apartments 086910279 15050 10 Seine Saint Denis, apartments 086910380 15060 10 Val de Marne, apartments 086910481 15070 10 Val d'Oise, apartments 080557385 12000 10 Petite Couronne, apartments 085102847 13000 10 Grande Couronne, apartments 086910683 15010 20 Seine et Marne, houses 086910784 15020 20 Yvelines, houses 086910885 15030 20 Essonne, houses 086910986 15040 20 Haut-de-Seine, houses 086911087 15050 20 Seine Saint Denis, houses 086911188 15060 20 Val de Marne, houses 086911289 15070 20 Val d'Oise, houses 086911390 15080 20 Petite Couronne, houses 086911491 12000 20 Grande Couronne, houses 080557486 20000 00 Province 080557587 20000 10 Province, apartments 067517959 20000 20 Province, houses 067518060 21000 10 Urban units > 10,000 inhabitants, apartments 067518161 21100 10 Urban units > 10,000 inhabitants, city center, apartments 067518262 21200 10 Urban units > 10,000 inhabitants, suburbs, apartments 080557688 22000 10 Urban units < 10,000 inhabitants and rural areas, apartments 087986777 30000 00 Provence-Alpes-Côte d'Azur 087986878 30000 10 Provence-Alpes-Côte d'Azur, apartments 087986979 30000 20 Provence-Alpes-Côte d'Azur, houses 087987080 31000 10 Marseilles, urban unit, apartments 087987181 40000 00 Rhône-Alpes 087987282 40000 10 Rhône-Alpes, apartments 087987383 40000 20 Rhône-Alpes, houses 087987484 41000 10 Lyons, urban unit, apartments NB : Petite Couronne: the first circle of outskirts of Paris (Haut-de-Seine, Seine-Saint-Denis, Val-de- Marne). Grande Couronne: the rest of Ile-de-France, further from Paris (Essonne, Seine-et-Marne, Yvelines, Val d'Oise). Province: all other départements of metropolitan France, except Corsica. 16
Table 2. Rate of non-response (percent) Zone Surface Number Time of Number of Number of Level or Lift of rooms construction garages bathrooms number of parking lots levels Province House 40.2 6.3 27.4 40.6 11.5 8.6 - Apartment 9.0 1.8 25.1 56.2 7.0 4.6 58.9 Ile de France House 46.3 0.1 51.3 - - 0.2 - Petite Couronne 66 0.2 74.0 - - 0.2 - Grande Couronne 38.5 0.1 42.4 - - 0.2 - Apartment 12.4 1.2 16.7 - - - 61.1 Paris 3.4 2.1 10.6 - - - 66.4 Petite couronne 22.7 0.7 21.8 - - - 64.1 Grande couronne 10.7 0.6 18.8 - - - 47.8 NB : See note of table 1. The rates are computed on the reference stock. In Ile-de-France, for garages, bathrooms non-responses are mixed up with `no bathroom' or `no garage'. 17
Table 3. Treatment of non-responses Type of non-response Action Method, if imputation Price rejected Surface and number of rooms rejected Surface imputed econometric Number of rooms rejected (Province) imputed (Ile-de- imputed from the surface; rejected France) from the estimation park, included in reference park Type (house or apartment) rejected Lift imputed Yes Level imputed Ground Floor Bathroom imputed No bathroom Garage, parking lot imputed No garage, no parking Time of construction 'Non-response' is a category Type of buyer imputed Private individual or SCI Occupation imputed Not rented Destination imputed Habitation, full property Surface of plot (for houses) rejected 18
Table 4. Number of strata, neighborhoods, and size of reference and estimation parks Index Number of Number of Size of reference park Size of estimation strata neighborhoods park Ile-de-France (total) 62 230 382 111 342 947 Apartments 55 205 262 102 277 120 Houses 7 25 120 009 65 827 Province (total) 234 1 125 837 552 848 286 Apartments 88 509 431 326 431 713 UU > 10 000 inhab. 74 410 356 133 203 276 city center 57 297 276 395 87 283 suburbs 17 113 79 738 115 993 UU< 10 000 inhab, rural 14 99 75 193 228 437 Houses 146 616 406226 416 573 Total 296 1355 1 219 663 1 191 233 NB: For houses in Ile-de-France, properties for which size is imputed are excluded from the estimation park; they are not excluded in the Province. 19
Table 5. Example of hedonic model: Houses in Seine-et-Marne Variables Coefficient Standard-error P-value constant 10.963 0.009 0.000 Surface in square meters 0.005 0.000 0.000 Plot in hectares 0.163 0.000 0.000 Year 1998 Reference Year 1999 0.056 0.004 0.000 Year 2000 0.123 0.005 0.000 Year 2001 0.191 0.005 0.000 Quarter 1 -0.049 0.005 0.000 Quarter 2 -0.020 0.004 0.000 Quarter 3 0.012 0.004 0.006 Quarter 4 Reference Neighborhood 1: Meaux Reference Neighborhood 2: Melun 0.087 0.005 0.000 Neighborhood 3: Provins -0.189 0.006 0.000 Neighborhood 4: Fontainebleau 0.053 0.005 0.000 Neighborhood 5; Torcy 0.211 0.005 0.000 Built before 1913 -0.074 0.006 0.000 1914 – 1947 0.020 0.006 0.002 1948 – 1969 0.047 0.006 0.000 1970 – 1980 0.034 0.005 0.000 Built after 1980 0.028 0.005 0.000 Date unknown Reference 0 bathroom -0.305 0.008 0.000 1 bathroom Reference 2 bathrooms 0.077 0.004 0.000 3 bathrooms or more 0.140 0.012 0.000 0 garage -0.070 0.004 0.000 1 garage Reference 2 garages or more 0.063 0.006 0.000 1 level 0.032 0.004 0.000 2 levels Reference 3 levels or more -0.016 0.006 0.008 3 rooms or less -0.107 0.005 0.000 4 rooms Reference 5 rooms 0.026 0.004 0.000 6 rooms 0.058 0.006 0.000 7 rooms or more 0.054 0.007 0.000 Number of observations 18,697 R square 0.696 20
Table 6. Example of hedonic model: Appartments in the city of Dijon Variables Coefficient Standard-error P-value Constant 8.792 0.034 0.000 Year 1998 reference Year 1999 0.005 0.010 0.640 Year 2000 0.018 0.010 0.070 Year 2001 0.066 0.010 0.000 Quarter 1 -0.013 0.010 0.218 Quarter 2 -0.001 0.009 0.897 Quarter 3 0.019 0.009 0.045 Quarter 4 reference Neighborhood 1: Historical center reference 2: Victor Hugo-Montchapet -0.015 0.029 0.603 3: Clémenceau-30 Octobre-Voltaire -0.099 0.032 0.002 4: Wilson-Auxonne-Parc -0.023 0.034 0.502 5: Facultés -0.108 0.034 0.001 6: Arsenal-Castel-Moulins -0.115 0.030 0.000 7: Hôpital général-Bourroche-Montagne -0.077 0.030 0.010 8: Fontaine’Ouche-Gare Berbisey -0.447 0.035 0.000 Built before 1850 0.154 0.026 0.000 1850-1913 0.074 0.020 0.000 1914 – 1947 0.074 0.018 0.000 1948 – 1969 reference 1970 – 1980 0.044 0.010 0.000 1981-1991 0.154 0.011 0.000 1992-2010 0.242 0.017 0.000 Date unknown 0.066 0.013 0.000 0 bathroom -0.069 0.043 0.114 1 bathroom reference 2 bathrooms or more 0.017 0.016 0.307 0 garage reference 1 garage 0.070 0.010 0.000 2 garages or more 0.159 0.016 0.000 Ground floor reference 1st floor 0.036 0.010 0.001 2d floor 0.061 0.011 0.000 3d floor 0.040 0.011 0.000 4th floor and more no lift 0.037 0.011 0.001 4th floor and more with lift -0.054 0.023 0.020 1 room 0.141 0.061 0.020 2 rooms 0.017 0.041 0.677 3 rooms reference 4 rooms 0.040 0.037 0.280 5 rooms or more 0.073 0.038 0.053 Surface/room of 1-room apartments 30 m² -0.071 0.023 0.002 Surface/room of 2-room 24 m² 0.001 0.016 0.954 Surface/room of 3-room
Surface/room of 3-room 18-22 m² reference Surface/room of 3-room >22 m² 0.029 0.014 0.035 Surface/room of >= 4-room = 4-room 17-21 m² reference Surface/room of >= 4-room >21 m² 0.043 0.013 0.001 Fair reference Some rehabilitation -0.080 0.011 0.000 To be renovated -0.182 0.023 0.000 unknown quality -0.014 0.009 0.129 Presence of a cellar unknown 0.139 0.069 0.043 no cellar reference 1 cellar or more -0.005 0.011 0.677 No terrace, no balcony reference terrace or balcony 0.042 0.008 0.000 1-room in neighborhood 2 -0.095 0.067 0.152 2 rooms in neighborhood 2 0.033 0.045 0.468 4 rooms in neighborhood 2 -0.036 0.040 0.368 5 rooms or more in neighborhood 2 -0.004 0.042 0.918 1-room in neighborhood 3 -0.115 0.071 0.106 2 rooms in neighborhood 3 0.052 0.048 0.277 4 rooms in neighborhood 3 -0.003 0.045 0.941 5 rooms or more in neighborhood 3 -0.111 0.049 0.024 1-room in neighborhood 4 -0.024 0.073 0.741 2 rooms in neighborhood 4 -0.036 0.051 0.484 4 rooms in neighborhood 4 0.048 0.049 0.330 5 rooms or more in neighborhood 4 0.028 0.052 0.581 1-room in neighborhood 5 0.039 0.067 0.561 2 rooms in neighborhood 5 0.075 0.050 0.131 4 rooms in neighborhood 5 -0.059 0.046 0.197 5 roomsor more in neighborhood 5 -0.046 0.053 0.394 1-room in neighborhood 6 -0.110 0.065 0.088 2 rooms in neighborhood 6 -0.020 0.046 0.658 4 rooms in neighborhood 6 -0.068 0.041 0.092 5 rooms or more in neighborhood 6 -0.128 0.045 0.005 1-room in neighborhood 7 -0.147 0.066 0.027 2 rooms in neighborhood 7 -0.009 0.046 0.852 4 rooms in neighborhood 7 -0.090 0.040 0.025 5 rooms or more in neighborhood 7 -0.103 0.051 0.041 1-room in neighborhood 8 0.240 0.077 0.002 2 rooms in neighborhood 8 0.070 0.052 0.179 4 rooms in neighborhood 8 -0.163 0.046 0.000 5 rooms or more in neighborhood 8 -0.086 0.052 0.097 Number of observations 2 215 R square 0.63 22
260 240 220 Lyon Marseille 200 Toulouse Rennes Index (2000 Q4=100) 180 Strasbourg 160 140 120 100 80 60 -4 -2 -4 -2 -4 -2 -4 -2 -4 -2 -4 -2 -4 -2 -4 -2 -4 -2 -4 -2 -4 -2 -4 94 95 95 96 96 97 97 98 98 99 99 00 00 01 01 02 02 03 03 04 04 05 05 19 19 19 19 19 19 19 19 19 19 19 20 20 20 20 20 20 20 20 20 20 20 20 Date 260 240 220 Nice 200 Nantes 180 Bordeaux 160 Lille 140 120 100 80 60 D at e Figure 1. The indexes for apartments in 9 cities (villes-centres) 23
Figure 2. The main Notaires-INSEE indexes : apartments 220 200 Paris 180 Petite Couronne 160 Grande Couronne 140 Province 120 100 80 60 -3 -2 -1 -4 -3 -2 -1 -4 -3 -2 -1 -4 -3 -2 -1 -4 -3 -2 -1 -4 91 92 93 93 94 95 96 96 97 98 99 99 00 01 02 02 03 04 05 05 19 19 19 19 19 19 19 19 19 19 19 19 20 20 20 20 20 20 20 20 Index, base 100 in 2000 Q4. 8 7 Paris 6 Petite Couronne 5 Grande Couronne 4 Province 3 2 1 0 -3 -2 -1 -4 -3 -2 19 1 -4 -3 -2 -1 -4 -3 -2 -1 -4 -3 -2 -1 -4 -1 - 91 92 93 93 94 95 96 96 97 98 99 99 00 01 02 02 03 04 05 05 19 19 19 19 19 19 19 19 19 19 19 20 20 20 20 20 20 20 20 -2 -3 -4 -5 -6 -7 -8 Rate of increase of the index in percent. 24
19 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 91 19 -3 91 60 80 100 120 140 160 180 200 19 - 92 -1 19 3 19 92 92 19 -3 - 93 19 2 -1 93 19 - 93 -3 19 1 19 93 94 19 -1 - 94 19 4 -3 94 Ile-de- 19 - France 95 -1 19 3 Province 19 95 95 -3 - Index, base 100 in 2000 Q4. 19 19 2 96 -1 96 19 - 96 -3 19 1 19 96 97 19 -1 - 97 19 4 -3 97 19 - 98 Rate of increase of the index in percent. -1 19 3 19 98 98 19 -3 - 99 19 2 Province -1 99 19 - 99 -3 19 1 20 Ile-de-France 00 99 20 -1 - 00 20 4 -3 00 20 - 01 Figure 3. The main Notaires-INSEE indexes : houses -1 20 3 20 01 01 20 -3 - 02 20 2 -1 02 20 - 02 -3 20 1 20 02 03 20 -1 - 03 20 4 -3 03 20 - 04 -1 20 3 20 04 04 -3 - 20 20 2 05 -1 05 20 - 05 -3 20 1 20 05 06 -1 -4 25
You can also read