2 Developer's Frequently Asked Questions (FAQ) for PostgreSQL
4 Last updated: Mon Feb 22 17:15:06 EST 1999
6 Current maintainer: Bruce Momjian (maillist@candle.pha.pa.us)
8 The most recent version of this document can be viewed at the
9 postgreSQL Web site, http://postgreSQL.org.
10 _________________________________________________________________
14 1) What tools are available for developers?
15 2) What books are good for developers?
16 3) Why do we use palloc() and pfree() to allocate memory?
17 4) Why do we use Node and List to make data structures?
18 5) How do I add a feature or fix a bug?
19 6) How do I download/update the current source tree?
20 7) How do I test my changes?
21 7) I just added a field to a structure. What else should I do?
22 8) Why are table, column, type, function, view names sometimes
23 referenced as Name or NameData, and sometimes as char *?
24 9) How do I efficiently access information in tables from the backend
27 _________________________________________________________________
29 1) What tools are available for developers?
31 Aside from the User documentation mentioned in the regular FAQ, there
32 are several development tools available. First, all the files in the
33 /tools directory are designed for developers.
34 RELEASE_CHANGES changes we have to make for each release
35 SQL_keywords standard SQL'92 keywords
36 backend description/flowchart of the backend directorie
38 ccsym find standard defines made by your compiler
39 entab converts tabs to spaces, used by pgindent
40 find_static finds functions that could be made static
41 find_typedef get a list of typedefs in the source code
42 make_ctags make vi 'tags' file in each directory
43 make_diff make *.orig and diffs of source
44 make_etags make emacs 'etags' files
45 make_keywords.README make comparison of our keywords and SQL'92
46 make_mkid make mkid ID files
47 mkldexport create AIX exports file
48 pgindent indents C source files
50 Let me note some of these. If you point your browser at the
51 file:/usr/local/src/pgsql/src/tools/backend/index.html directory, you
52 will see few paragraphs describing the data flow, the backend
53 components in a flow chart, and a description of the shared memory
54 area. You can click on any flowchart box to see a description. If you
55 then click on the directory name, you will be taken to the source
56 directory, to browse the actual source code behind it. We also have
57 several README files in some source directories to describe the
58 function of the module. The browser will display these when you enter
59 the directory also. The tools/backend directory is also contained on
60 our web page under the title How PostgreSQL Processes a Query.
62 Second, you really should have an editor that can handle tags, so you
63 can tag a function call to see the function definition, and then tag
64 inside that function to see an even lower-level function, and then
65 back out twice to return to the original function. Most editors
66 support this via tags or etags files.
68 Third, you need to get mkid from ftp.postgresql.org. By running
69 tools/make_mkid, an archive of source symbols can be created that can
70 be rapidly queried like grep or edited. Others prefer glimpse.
72 make_diff has tools to create patch diff files that can be applied to
75 pgindent will format source files to match our standard format, which
76 has four-space tabs, and an indenting format specified by flags to the
77 your operating system's utility indent.
79 pgindent is run on all source files just before each beta test period.
80 It auto-formats all source files to make them consistent. Comment
81 blocks that need specific line breaks should be formatted as block
82 comments, where the comment starts as /*------. These comments will
83 not be reformatted in any way.
85 2) What books are good for developers?
87 I have four good books, An Introduction to Database Systems, by C.J.
88 Date, Addison, Wesley, A Guide to the SQL Standard, by C.J. Date, et.
89 al, Addison, Wesley, Fundamentals of Database Systems, by Elmasri and
90 Navathe, and Transaction Processing, by Jim Gray, Morgan, Kaufmann
92 There is also a database performance site, with a handbook on-line
93 written by Jim Gray at http://www.benchmarkresources.com.
95 3) Why do we use palloc() and pfree() to allocate memory?
97 palloc() and pfree() are used in place of malloc() and free() because
98 we automatically free all memory allocated when a transaction
99 completes. This makes it easier to make sure we free memory that gets
100 allocated in one place, but only freed much later. There are several
101 contexts that memory can be allocated in, and this controls when the
102 allocated memory is automatically freed by the backend.
104 4) Why do we use Node and List to make data structures?
106 We do this because this allows a consistent way to pass data inside
107 the backend in a flexible way. Every node has a NodeTag which
108 specifies what type of data is inside the Node. Lists are groups of
109 Nodes chained together as a forward-linked list.
111 Here are some of the List manipulation commands:
114 return the data at list element i.
117 return the next list element after i.
120 loop through list, assigning each list element to i. It is
121 important to note that i is a List *, not the data in the List
122 element. You need to use lfirst(i) to get at the data. Here is
123 a typical code snipped that loops through a List containing Var
124 *'s and processes each one:
131 Var *var = lfirst(i);
133 /* process var here */
137 add node to the front of list, or create a new list with node
141 add node to the end of list. This is more expensive that lcons.
144 Concat list2 on to the end of list1.
147 return the length of the list.
150 return the i'th element in list.
153 There are integer versions of these: lconsi, lappendi, nthi.
154 List's containing integers instead of Node pointers are used to
155 hold list of relation object id's and other integer quantities.
157 You can print nodes easily inside gdb. First, to disable output
158 truncation when you use the gdb print command:
160 (gdb) set print elements 0
162 Instead of printing values in gdb format, you can use the next two
163 commands to print out List, Node, and structure contents in a verbose
164 format that is easier to understand. List's are unrolled into nodes,
165 and nodes are printed in detail. The first prints in a short format,
166 and the second in a long format:
168 (gdb) call print(any_pointer)
169 (gdb) call pprint(any_pointer)
171 The output appears in the postmaster log file, or on your screen if
172 you are running a backend directly without a postmaster.
174 5) How do I add a feature or fix a bug?
176 The source code is over 250,000 lines. Many problems/features are
177 isolated to one specific area of the code. Others require knowledge of
178 much of the source. If you are confused about where to start, ask the
179 hackers list, and they will be glad to assess the complexity and give
180 pointers on where to start.
182 Another thing to keep in mind is that many fixes and features can be
183 added with surprisingly little code. I often start by adding code,
184 then looking at other areas in the code where similar things are done,
185 and by the time I am finished, the patch is quite small and compact.
187 When adding code, keep in mind that it should use the existing
188 facilities in the source, for performance reasons and for simplicity.
189 Often a review of existing code doing similar things is helpful.
191 6) How do I download/update the current source tree?
193 There are several ways to obtain the source tree. Occasional
194 developers can just get the most recent source tree snapshot from
195 ftp.postgresql.org. For regular developers, you can use CVS. CVS
196 allows you to download the source tree, then occasionally update your
197 copy of the source tree with any new changes. Using CVS, you don't
198 have to download the entire source each time, only the changed files.
199 Anonymous CVS does not allows developers to update the remote source
200 tree, though privileged developers can do this. There is a CVS FAQ on
201 our web site that describes how to use remote CVS. You can also use
202 CVSup, which has similarly functionality, and is available from
205 To update the source tree, there are two ways. You can generate a
206 patch against your current source tree, perhaps using the make_diff
207 tools mentioned above, and send them to the patches list. They will be
208 reviewed, and applied in a timely manner. If the patch is major, and
209 we are in beta testing, the developers may wait for the final release
210 before applying your patches.
212 For hard-core developers, Marc(scrappy@postgresql.org) will give you a
213 Unix shell account on postgresql.org, so you can use CVS to update the
214 main source tree, or you can ftp your files into your account, patch,
215 and cvs install the changes directly into the source tree.
217 6) How do I test my changes?
219 First, use psql to make sure it is working as you expect. Then run
220 src/test/regress and get the output of src/test/regress/checkresults
221 with and without your changes, to see that your patch does not change
222 the regression test in unexpected ways. This practice has saved me
223 many times. The regression tests test the code in ways I would never
224 do, and has caught many bugs in my patches. By finding the problems
225 now, you save yourself a lot of debugging later when things are
226 broken, and you can't figure out when it happened.
228 7) I just added a field to a structure. What else should I do?
230 The structures passing around from the parser, rewrite, optimizer, and
231 executor require quite a bit of support. Most structures have support
232 routines in src/backend/nodes used to create, copy, read, and output
233 those structures. Make sure you add support for your new field to
234 these files. Find any other places the structure may need code for
235 your new field. mkid is helpful with this (see above).
237 8) Why are table, column, type, function, view names sometimes referenced as
238 Name or NameData, and sometimes as char *?
240 Table, column, type, function, and view names are stored in system
241 tables in columns of type Name. Name is a fixed-length,
242 null-terminated type of NAMEDATALEN bytes. (The default value for
243 NAMEDATALEN is 32 bytes.)
244 typedef struct nameData
246 char data[NAMEDATALEN];
248 typedef NameData *Name;
250 Table, column, type, function, and view names that come into the
251 backend via user queries are stored as variable-length,
252 null-terminated character strings.
254 Many functions are called with both types of names, ie. heap_open().
255 Because the Name type is null-terminated, it is safe to pass it to a
256 function expecting a char *. Because there are many cases where
257 on-disk names(Name) are compared to user-supplied names(char *), there
258 are many cases where Name and char * are used interchangeably.
260 9) How do I efficiently access information in tables from the backend code?
262 You first need to find the tuples(rows) you are interested in. There
263 are two ways. First, SearchSysCacheTuple() and related functions allow
264 you to query the system catalogs. This is the preferred way to access
265 system tables, because the first call to the cache loads the needed
266 rows, and future requests can return the results without accessing the
267 base table. Some of the caches use system table indexes to look up
268 tuples. A list of available caches is located in
269 src/backend/utils/cache/syscache.c.
270 src/backend/utils/cache/lsyscache.c contains many column-specific
271 cache lookup functions.
273 The rows returned are cached-owned versions of the heap rows. They are
274 invalidated when the base table changes. Because the cache is local to
275 each backend, you may use the pointer returned from the cache for
276 short periods without making a copy of the tuple. If you send the
277 pointer into a large function that will be doing its own cache
278 lookups, it is possible the cache entry may be flushed, so you should
279 use SearchSysCacheTupleCopy() in these cases, and pfree() the tuple
282 If you can't use the system cache, you will need to retrieve the data
283 directly from the heap table, using the buffer cache that is shared by
284 all backends. The backend automatically takes care of loading the rows
285 into the buffer cache.
287 Open the table with heap_open(). You can then start a table scan with
288 heap_beginscan(), then use heap_getnext() and continue as long as
289 HeapTupleIsValid() returns true. Then do a heap_endscan(). Keys can be
290 assigned to the scan. No indexes are used, so all rows are going to be
291 compared to the keys, and only the valid rows returned.
293 You can also use heap_fetch() to fetch rows by block number/offset.
294 While scans automatically lock/unlock rows from the buffer cache, with
295 heap_fetch(), you must pass a Buffer pointer, and ReleaseBuffer() it
296 when completed. Once you have the row, you can get data that is common
297 to all tuples, like t_self and t_oid, by mererly accessing the
298 HeapTuple structure entries. If you need a table-specific column, you
299 should take the HeapTuple pointer, and use the GETSTRUCT() macro to
300 access the table-specific start of the tuple. You then cast the
301 pointer as a Form_pg_proc pointer if you are accessing the pg_proc
302 table, or Form_pg_type if you are accessing pg_type. You can then
303 access the columns by using a structure pointer:
305 ((Form_pg_class) GETSTRUCT(tuple))->relnatts
307 You should not directly change live tuples in this way. The best way
308 is to use heap_tuplemodify() and pass it your palloc'ed tuple, and the
309 values you want changed. It returns another palloc'ed tuple, which you
310 pass to heap_replace(). You can delete tuples by passing the tuple's
311 t_self to heap_destroy(). Remember, tuples can be either system cache
312 versions, which may go away soon after you get them, buffer cache
313 version, which will go away when you heap_getnext(), heap_endscan, or
314 ReleaseBuffer(), in the heap_fetch() case. Or it may be a palloc'ed
315 tuple, that you must pfree() when finished.
319 elog() is used to send messages to the front-end, and optionally
320 terminate the current query being processed. The first parameter is an
321 elog level of NOTICE, DEBUG, ERROR, or FATAL. NOTICE prints on the
322 user's terminal and the postmaster logs. DEBUG prints only in the
323 postmaster logs. ERROR prints in both places, and terminates the
324 current query, never returning from the call. FATAL terminates the
325 backend process. The remaining parameters of elog are a printf-style
326 set of parameters to print.
OSDN Git Service
