Hello Steve,

On Fri, Apr 30, 2010 at 6:11 PM, Steve Singer <ssinger at ca.afilias.info> wrote:
>>>> I will try to find out what log entry is causing the trouble.
>>>>
>>> Do the tables that these errors come from typically get large
>>> insert/update
>>> statements or small ones?
>>>
>>> (this is directed at anyone who's seen the issue)
>>
>> I'm still trying to isolate the issue a bit, but at the moment it
>> looks like this issue is triggered by a very large INSERT statement
>> (as in, a statement with a very large value);
>>
>> Regards,
>>
>
> Your issue with the larger values are being caused by an attempt at removing
> a memory leak checked into 2.0.3 that free's memory a bit too soon.  I will
> post a patch for this later today.   I think the INSERT statement is being
> corrupted causing invalid characters to be put in.

Ok, thanks for the information. It looks like you are correct. I have
done some investigation, and am receiving different errors right now,
but it appears to be caused by the same issue:

The pg_log entry says this:

---

LOG:  statement: insert into "_tsukku".sl_log_2     (log_origin,
log_txid, log_tableid,      log_actionseq, log_cmdtype,
log_cmddata) values     ('1', '22643', '121', '58', 'I',
'("id","state","ping","perc","last_monitor_update","input","output","hadoop_task_id")
values (''15324'',NULL,''2010-04-30
16:07:02.482649'',NULL,''2010-04-30
16:07:02.482649'',''/solatis/tsukku/store/requests/2010/04/30/'',''/solatis/tsukku/mapreduce/check_uncommitted_log_transactions/69efacb6-5472-11df-8fee-0050562fea49'',NULL)');
insert into "_tsukku".sl_log_2     (log_origin, log_txid, log_tableid,
     log_actionseq, log_cmdtype,      log_cmddata) values     ('1',
'22643', '122', '59', 'I', '("map_reduce_task_id","key","value")
values (''15324'',''solatis.tsukku.task.type'',''4'')');
insert into "_tsukku".sl_log_2     (log_origin, log_txid, log_tableid,
     log_actionseq, log_cmdtype,      log_cmddata) values     ('1',
'22643', '122', '60', 'I', '("map_reduce_task_id","key","value")
values (''15324'',''hadoop.pipes.executable'',''/solatis/tsukku/bin/mapreduce'')');
insert into "_tsukku".sl_log_2     (log_origin, log_txid, log_tableid,
     log_actionseq, log_cmdtype,      log_cmddata) values     ('1',
'22643', '122', '61', 'I', '("map_reduce_task_id","key","value")
values (''15324'',''mapred.child.java.opts'',''-Xmx256m'')');
insert into "_tsukku".sl_log_2     (log_origin, log_txid, log_tableid,
     log_actionseq, log_cmdtype,      log_cmddata) values     ('1',
'22643', '122', '62', 'I', '`dn');
insert into "_tsukku".sl_log_2     (log_origin, log_txid, log_tableid,
     log_actionseq, log_cmdtype,      log_cmddata) values     ('1',
'22643', '122', '63', 'I', '("map_reduce_task_id","key","value")
values (''15324'',''solatis.tsukku.task.log.begin_date'',''20100324'')');
insert into "_tsukku".sl_log_2     (log_origin, log_txid, log_tableid,
     log_actionseq, log_cmdtype,      log_cmddata) values     ('1',
'22643', '122', '64', 'I', '("map_reduce_task_id","key","value")
values (''15324'',''solatis.tsukku.task.log.end_date'',''20100430'')');

ERROR:  syntax error at or near "`" at character 830

---

So, when searching for the "`" character, we see this line:

insert into "_tsukku".sl_log_2     (log_origin, log_txid, log_tableid,
     log_actionseq, log_cmdtype,      log_cmddata) values     ('1',
'22643', '122', '62', 'I', '`dn');

Looking up that entry in the master database indeed brings up the
large INSERT statement. For what it's worth:

---
tsukku=# select char_length (log_cmddata) from _tsukku.sl_log_2 where
log_txid = 22643 and log_actionseq = 62;
 char_length
-------------
       60533
(1 row)
---

Thanks for your help. In the meantime, is there anything I can do to
work around this issue?

Regards,

Leon Mergen