summaryrefslogtreecommitdiff
path: root/src/backend/commands/copy.c
AgeCommit message (Collapse)Author
2005-05-06Convert some mulit-line comments in copy.c to single line, as appropriate.Bruce Momjian
2005-05-01Change CREATE TYPE to require datatype output and send functions to haveTom Lane
only one argument. (Per recent discussion, the option to accept multiple arguments is pretty useless for user-defined types, and would be a likely source of security holes if it was used.) Simplify call sites of output/send functions to not bother passing more than one argument.
2005-04-14Completion of project to use fixed OIDs for all system catalogs andTom Lane
indexes. Replace all heap_openr and index_openr calls by heap_open and index_open. Remove runtime lookups of catalog OID numbers in various places. Remove relcache's support for looking up system catalogs by name. Bulky but mostly very boring patch ...
2005-03-25Improve EXPLAIN ANALYZE to show the time spent in each trigger whenTom Lane
executing a statement that fires triggers. Formerly this time was included in "Total runtime" but not otherwise accounted for. As a side benefit, we avoid re-opening relations when firing non-deferred AFTER triggers, because the trigger code can re-use the main executor's ResultRelInfo data structure.
2005-03-16Revise TupleTableSlot code to avoid unnecessary construction and disassemblyTom Lane
of tuples when passing data up through multiple plan nodes. A slot can now hold either a normal "physical" HeapTuple, or a "virtual" tuple consisting of Datum/isnull arrays. Upper plan levels can usually just copy the Datum arrays, avoiding heap_formtuple() and possible subsequent nocachegetattr() calls to extract the data again. This work extends Atsushi Ogawa's earlier patch, which provided the key idea of adding Datum arrays to TupleTableSlots. (I believe however that something like this was foreseen way back in Berkeley days --- see the old comment on ExecProject.) A test case involving many levels of join of fairly wide tables (about 80 columns altogether) showed about 3x overall speedup, though simple queries will probably not be helped very much. I have also duplicated some code in heaptuple.c in order to provide versions of heap_formtuple and friends that use "bool" arrays to indicate null attributes, instead of the old convention of "char" arrays containing either 'n' or ' '. This provides a better match to the convention used by ExecEvalExpr. While I have not made a concerted effort to get rid of uses of the old routines, I think they should be deprecated and eventually removed.
2005-03-12Handle carriage returns and line feeds in COPY CSV mode.Bruce Momjian
Andrew Dunstan
2004-12-31Tag appropriate files for rc3PostgreSQL Daemon
Also performed an initial run through of upgrading our Copyright date to extend to 2005 ... first run here was very simple ... change everything where: grep 1996-2004 && the word 'Copyright' ... scanned through the generated list with 'less' first, and after, to make sure that I only picked up the right entries ...
2004-12-03> If it bothers you that much. I'd make a flag, cleared at the start ofBruce Momjian
> each COPY, and then where we test for CR or LF in CopyAttributeOutCSV, > if the flag is not set then set it and issue the warning. Andrew Dunstan
2004-11-06When implementing a coercion to a domain type with a combinedTom Lane
type-and-length coercion function, make sure that the coercion function is told the correct typmod. Fixes Kris Jurka's example of a domain over bit(N).
2004-10-29Fix failure to think clearly about encoding conversion errors in COPY.Tom Lane
We can't regurgitate the unconverted string as I first thought, because the elog.c mechanisms will assume the error message data is in the server encoding and attempt a reverse conversion. Eventually it might be worth providing a short-circuit path to support this, but for now the simplest solution is to abandon trying to report back the line contents after a conversion failure. Per bug report from Sil Lee, 27-Oct-2004.
2004-09-13Redesign query-snapshot timing so that volatile functions in READ COMMITTEDTom Lane
mode see a fresh snapshot for each command in the function, rather than using the latest interactive command's snapshot. Also, suppress fresh snapshots as well as CommandCounterIncrement inside STABLE and IMMUTABLE functions, instead using the snapshot taken for the most closely nested regular query. (This behavior is only sane for read-only functions, so the patch also enforces that such functions contain only SELECT commands.) As per my proposal of 6-Sep-2004; I note that I floated essentially the same proposal on 19-Jun-2002, but that discussion tailed off without any action. Since 8.0 seems like the right place to be taking possibly nontrivial backwards compatibility hits, let's get it done now.
2004-09-10Fire non-deferred AFTER triggers immediately upon query completion,Tom Lane
rather than when returning to the idle loop. This makes no particular difference for interactively-issued queries, but it makes a big difference for queries issued within functions: trigger execution now occurs before the calling function is allowed to proceed. This responds to numerous complaints about nonintuitive behavior of foreign key checking, such as http://archives.postgresql.org/pgsql-bugs/2004-09/msg00020.php, and appears to be required by the SQL99 spec. Also take the opportunity to simplify the data structures used for the pending-trigger list, rename them for more clarity, and squeeze out a bit of space.
2004-08-29Pgindent run for 8.0.Bruce Momjian
2004-08-29Update copyright to 2004.Bruce Momjian
2004-07-31Restructure error handling as recently discussed. It is now reallyTom Lane
possible to trap an error inside a function rather than letting it propagate out to PostgresMain. You still have to use AbortCurrentTransaction to clean up, but at least the error handling itself will cooperate.
2004-06-16Represent type-specific length coercion functions as pg_cast entries,Tom Lane
eliminating the former hard-wired convention about their names. Allow pg_cast entries to represent both type coercion and length coercion in a single step --- this is represented by a function that takes an extra typmod argument, just like a length coercion function. This nicely merges the type and length coercion mechanisms into something at least a little cleaner than we had before. Make use of the single- coercion-step behavior to fix integer-to-bit coercion so that coercing to bit(n) yields the rightmost n bits of the integer instead of the leftmost n bits. This should fix recurrent complaints about the odd behavior of this coercion. Clean up the documentation of the bit string functions, and try to put it where people might actually find it. Also, get rid of the unreliable heuristics in ruleutils.c about whether to display nested coercion steps; instead require parse_coerce.c to label them properly in the first place.
2004-06-06Infrastructure for I/O of composite types: arrange for the I/O routinesTom Lane
of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-05Tweak palloc/repalloc to allow zero bytes to be requested, as per recentTom Lane
proposal. Eliminate several dozen now-unnecessary hacks to avoid palloc(0). (It's likely there are more that I didn't find.)
2004-05-26Reimplement the linked list data structure used throughout the backend.Neil Conway
In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-04-21Change COPY CSV keyword to be:Bruce Momjian
FORCE QUOTE to force quotes FORCE NOT NULL to quote null input values
2004-04-19Prevent doubling of escapes when not in quote mode for CSV.Bruce Momjian
2004-04-19Complete TODO item:Bruce Momjian
o -Allow dump/load of CSV format This adds new keywords to COPY and \copy: CSV - enable CSV mode (comma separated variable) QUOTE - specify quote character ESCAPE - specify escape character FORCE - force quoting of specified column LITERAL - suppress null comparison for columns Doc changes included. Regression updates coming from Andrew.
2004-04-15Modify COPY for() loop to use attnum as a variable name, not 'i'.Bruce Momjian
2004-04-06The attached applied patch throws an error if the delimiter appears inBruce Momjian
the COPY NULL string: test=> copy pg_language to '/tmp/x' with delimiter '|'; COPY test=> copy pg_language to '/tmp/x' with delimiter '|' null '|x'; ERROR: COPY delimiter must not appear in the NULL specification test=> copy pg_language from '/tmp/x' with delimiter '|' null '|x'; ERROR: COPY delimiter must not appear in the NULL specification It also throws an error if it conflicts with the default NULL string: test=> copy pg_language to '/tmp/x' with delimiter '\\'; ERROR: COPY delimiter must not appear in the NULL specification test=> copy pg_language to '/tmp/x' with delimiter '\\' NULL 'x'; COPY
2004-02-10Restructure smgr API as per recent proposal. smgr no longer depends onTom Lane
the relcache, and so the notion of 'blind write' is gone. This should improve efficiency in bgwriter and background checkpoint processes. Internal restructuring in md.c to remove the not-very-useful array of MdfdVec objects --- might as well just use pointers. Also remove the long-dead 'persistent main memory' storage manager (mm.c), since it seems quite unlikely to ever get resurrected.
2004-01-28Review uses of IsUnderPostmaster, change some tests to look atTom Lane
whereToSendOutput instead because they are really inquiring about the correct client communication protocol. Update some comments. This is pointing towards supporting regular FE/BE client protocol in a standalone backend, per discussion a month or so back.
2004-01-26Ensure that close() and fclose() are checked for errors, at least inTom Lane
cases involving writes. Per recent discussion about the possibility of close-time failures on some filesystems. There is a TODO item for this, too.
2004-01-18Don't use %s-with-precision format spec to truncate data being displayedTom Lane
in a COPY error message. It seems that glibc gets indigestion if it is asked to truncate strings that contain invalid UTF-8 encoding sequences. vsnprintf will return -1 in such cases, leading to looping and eventual memory overflow in elog.c. Instead use our own, more robust pg_mbcliplen routine. I believe this problem accounts for several recent reports of unexpected 'out of memory' errors during COPY IN.
2003-11-29$Header: -> $PostgreSQL Changes ...PostgreSQL Daemon
2003-10-06Modify COPY FROM to match the null-value string against the column valueTom Lane
before it is de-backslashed, not after. This allows the null string \N to be reliably distinguished from the data value \N (which must be represented as \\N). Per bug report from Manfred Koizar ... but it's amazing this hasn't been reported before ... Also, be consistent about encoding conversion for null string: the form specified in the command is in the server encoding, but what is sent to/from client must be in client encoding. This never worked quite right before either.
2003-09-29Improve context display for failures during COPY IN, as recentlyTom Lane
discussed on pghackers.
2003-09-25Message editing: remove gratuitous variations in message wording, standardizePeter Eisentraut
terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-08-28Fix stupid typo (mine I suppose) in CopyGetData. Per report from Dave Cramer.Tom Lane
2003-08-13libpq failed to cope with COPY FROM STDIN if the command was issuedTom Lane
via extended query protocol, because it sends Sync right after Execute without realizing that the command to be executed is COPY. There seems to be no reasonable way for it to realize that, either, so the best fix seems to be to make the backend ignore Sync during copy-in mode. Bit of a wart on the protocol, but little alternative. Also, libpq must send another Sync after terminating the COPY, if the command was issued via Execute.
2003-08-08Another pgindent run with updated typedefs.Bruce Momjian
2003-08-04Update copyrights to 2003.Bruce Momjian
2003-08-04pgindent run.Bruce Momjian
2003-08-01Adjust 'permission denied' messages to be more useful and consistent.Tom Lane
2003-07-22Error message editing in backend/libpq, backend/postmaster, backend/tcop.Tom Lane
Along the way, fix some logic problems in pgstat_initstats, notably the bogus assumption that malloc returns zeroed memory.
2003-07-21Error message editing in backend/executor.Tom Lane
2003-07-20Another round of error message editing, covering backend/commands/.Tom Lane
2003-05-16Remove use of geteuid under Win32.Bruce Momjian
2003-05-09COPY BINARY uses the new binary I/O routines. Update a few more datatypesTom Lane
so that COPY BINARY regression test passes.
2003-05-09Implement new-protocol binary I/O support in DataRow, Bind, and FunctionCallTom Lane
messages. Binary I/O is now up and working, but only for a small set of datatypes (integers, text, bytea).
2003-05-08Update 3.0 protocol support to match recent agreements about how toTom Lane
handle multiple 'formats' for data I/O. Restructure CommandDest and DestReceiver stuff one more time (it's finally starting to look a bit clean though). Code now matches latest 3.0 protocol document as far as message formats go --- but there is no support for binary I/O yet.
2003-04-25COPY and pg_dump failed to cope with zero-column tables. Fix 'em.Tom Lane
2003-04-24Infrastructure for upgraded error reporting mechanism. elog.c isTom Lane
rewritten and the protocol is changed, but most elog calls are still elog calls. Also, we need to contemplate mechanisms for controlling all this functionality --- eg, how much stuff should appear in the postmaster log? And what API should libpq expose for it?
2003-04-22Another round of protocol changes. Backend-to-frontend messages now allTom Lane
have length words. COPY OUT reimplemented per new protocol: it doesn't need \. anymore, thank goodness. COPY BINARY to/from frontend works, at least as far as the backend is concerned --- libpq's PQgetline API is not up to snuff, and will have to be replaced with something that is null-safe. libpq uses message length words for performance improvement (no cycles wasted rescanning long messages), but not yet for error recovery.
2003-04-19Allow \r and \r\n termination for COPY files.Bruce Momjian
Output \r\n termination on Win32. Disallow literal carriage return as a data value, backslash-carriage-return and \r still allowed. Doc changes already committed.
2003-04-19Add pipe parameter to COPY function to allow proper line termination.Bruce Momjian