ISUG Technical Journal Sep/Oct 2008 is out! Includes my “Perl in the Shell” article

The ISUG Technical Journal Sep/Oct 2008 is out.

This issue contains:

  • Sybase IQ and Sybase WorkSpace Part II By Mike Crocker
  • Perl in the shell By Jason Froebe
  • Taming the process nightmare: Consolidating a Horde of Process Diagrams with PowerDesigner By Mike Nicewarner
  • Information overload: Weapons-grade data analysis comes of age By Teresa Foster
  • Maximizing PowerBuilder and SQLAnywhere Part III SQLAnywhere, a PowerBuilder Developer’s perspective
    By Chris Pollach
  • select random (stuff) from SYBASE Command shell by proxy By Bill Grant
  • Mobile Enterprise Everywhere: Taking Sybase on the road By Adrian Bridgwater
  • Sybase TechWave 2008: Time for the technology tsunami

You need to be an ISUG member to receive the ISUG Technical Journal.  Join ISUG today!

International Sybase User Group

International Sybase User Group

Share Button

What is with the ISUG Enhancements versus Sybase CR/Bugs on the sybase-product-futures mailing list?

In the MSA and System transactions performedby maint user thread on the sybase-product-futures mailing list, Jeff Tallman mentions thatInternational Sybase Users Group someone needs to create an ISUG enhancement or a Sybase change request (CR).  Mike Harrold let everyone know that even through creating a Sybase CR is put in the Sybase engineering queue, it doesn’t mean that an engineer will actually look at it let alone have product management approve it to be implemented.

Jeff hits the nail on the head.  You have a voice with the ISUG enhancements process. Use it.  And yes, you have to be paid member to vote. Sorry, but we are a non-profit and your membership fees keep us running!  You can vote as a Basic Member – you do not need to join at the higher membership levels (but you should so you can read all the wonderful articles Jeff writes for us).  The more votes, the easier it is for engineering to justify spending staff resources on it when they go to management.  Encourage all of your peers to join and vote as well.  If you work for a business with several Sybase users, convince your boss to pay for a corporate membership and get all your colleagues to vote as well. 🙂
Sybase
Regards,

Mike Harrold
Executive Director

and

Having an “internal enhancement” (aka a CR) doesn’t mean it gets implemented.  It means there’s a CR for it.  It might be a great idea, but without backing (and a user request + an engineer doesn’t mean it has backing; look how long it took for UDFs, mathematical functions, etc.) it doesn’t get allocated any development resources.  Without development resources, it doesn’t get implemented.

Bottom line, a CR doesn’t mean it’s a “planned” feature.

Regards,

Mike Harrold
Executive Director
International Sybase User Group

I would recommend voting for the ISUG enhancements (this week).  I and several other ISUG Board members will be going through the ISUG enhancements this weekend to present a subset to Sybase product management at TechWave next month.

I haven’t seen many ISUG enhancements being requested lately or voted upon for that matter.  The higher the votes for a particular enhancement, the more likely Sybase will pick it up.

So please, everyone go vote on the ISUG enhancements! 🙂

Share Button

Sybase Replication Server: Out of Mutexes? wtf?

For those of us that use Sybase’s Replication Server, we have long ago been pacified into believing that there really isn’t much that can be done with Replication Server’s quirks but to endure them.

In today’s edition of the Sybase RepServer Quirks we take a look at the “out of mutexes” error.  A mutex is an exclusive lock on a resource.  If you receive an error about RepServer running out of mutexes, you just don’t have enough defined through RepServer.  Simple huh?

Of course it is.  Ahh..  but then how do how many mutexes your particular Sybase RepServer needs?  How many grains of sand are on the moon?  According to Sybase you should know this already but since you can’t read minds (I hope you can’t because I’m thinking of donuts and not RepServer), I’ll pass on what Sybase is saying when pressed about it (Greg Carter @ Sybase):

> Just to elaborate a bit more on this question of the number of mutexes; as
> you probably know the mutex requirements for RepServer increased
> dramatically with the SMP feature. I have since struggled to come up with a
> formula for estimating mutex requirements so that you may properly set the
> “num_mutexes” configuration. While in the latest iteration of this formula
> I have satisfied myself that all mutexes have been accounted for, still the
> estimate it provides seems to fall short in some cases.
>
> Recent investigations by Connectivity seem to indicate that the problem may
> not be with sizing mutex requirements, but rather with sizing message queue
> (“num_msgqueues”) requirements. It appears that Open Server may be using
> the total of the settings for “num_mutexes” and “num_msgqueues” as the
> upper bound for the creation of these two objects together. So it may be
> that even though “num_mutexes” has been sized properly, if “num_msgqueues”
> is too low then you may see a message regarding the failure to create a
> mutex or the failure to create a message queue depending on which one was
> being created at the time that upper bound was surpassed.
>
> The moral here is that until Open Server resolves this issue you need to
> verify the sizing of both “num_mutexes” and “num_msgqueues” in the event
> that either error message appears since you can not rely on the message to
> indicate which one is low.
>
> For your convenience I’ll include here the latest formulas for estimating
> mutex and message queue requirements. Note that the one for mutexes may not
> agree completely with the one that is given in the 12.6 SMP White Paper – I
> have not compared them.
>
> Mutex requirements for the optimized binary:
> num_mutexes = 75 + Num(partitions) + 4*Num(DSI/S) + 3*Num(DSI/E) +
> 2*Num(Dist) + 2*Num(RepAgent Exec) + 2*Num(RSI User) + 5*Num(Queues) +
> 5*Num(SQT Cache) + Num(rs_subscriptions rows) + Num(RSSD tables) +
> Setting(cm_max_connections) +
> 2*MAX(Admin connections) + 2*Num(Other Connections) + Num(Origins) +
> 2*Num(Threads) + MAX(subscription (de)mat)
>
> Where
> – “Other Connections” are connection to this RepServer including ID Server
> connections, RSM connections, etc.
> – “Origins” are the different origins (or primary databases) that could
> possibly have transactions flowing through this RepServer, whether by a
> RepAgent or by a route (intermediate included)
> – “Threads” includes every thread RepServer may start. These are the
> “Global” thread, the “Initialization” thread, threads for each of the
> daemons (dAIO, dCM, dVersion, dRec, dSub, dStats, dAlarm), RepAgent User
> threads, SQM Writer threads, SQT threads, Distributor threads, DSI/S and
> DSI/E threads, RSI User and RSI threads.
> – “subscription (de)mat” is the number of asynchronous subscription
> management requests for materializing or dematerializing that may be taking
> place at any moment.
>
> Note: For the diagnostic binary you will need to double the figure
> determined with the above formula.
>
> Message queue requirements for the optimized and diagnostic binary:
> num_msgqueues = 10 + Num(DSI/S) + Num(DSI/E) + Num(Queues) + Num(Dist)
>
> Thanks,
> G.Carter

Neither this explanation or the two equations are anywhere in the manuals.  I’ve opened feature request 485482 for RepServer to handle this automagically as there really is NO NEED for a RepServer admin to have to worry about this.  If you are or ever have run into this problem, give Sybase a holler and tell them to fix this bug.

Share Button

FW Ken Rearick: repserver exception list/deleter

My good friend Ken Rearick wrote a rather clever little stored procedure for managing all the exceptions that are raised in the life of a Sybase Replication Server.  He posted this on sybase.public.rep-server a few days ago:

This is a rewriten rs_delexception script that will list out all exceptions and then step through deleting all of them. As this uses the same procedure for deleting the entries it does not cause any problems with integerity in the RSSD — Ken Rearick

IF OBJECT_ID(‘dbo.rs_del_all_exception’) IS NOT NULL
BEGIN
    DROP PROCEDURE dbo.rs_del_all_exception
    IF OBJECT_ID(‘dbo.rs_del_all_exception’) IS NOT NULL
        PRINT ‘< << FAILED DROPPING PROCEDURE dbo.rs_del_all_exception >>> >>>’
    ELSE
        PRINT ‘< << DROPPED PROCEDURE dbo.rs_del_all_exception >>>’
END
go

create proc rs_del_all_exception
@xact int = NULL

as

declare @systran binary(8)
declare @cnt int, @err int
declare @ccnt char(8)
declare @cxact char(8)
declare @rsname varchar(30)
declare @cmdcount char(9)
declare @msg varchar(255)
declare @tab_name varchar(30)
declare @row_cnt int

set nocount on

/* find RS name */

select @rsname = charvalue
from   rs_config
where  optionname = "oserver"

/* Build temp table #tab1 */

create table #tab1
(    orig_site varchar(30),
    orig_db   varchar(30),
    orig_user varchar(30),
    orig_time datetime,
    error_site varchar(30),
    error_db varchar(30),
    log_time datetime,
    reccount int null,
    sys_trans_id binary(8),
    app_usr varchar(30)
)

insert into #tab1 (orig_site, orig_db,
           orig_user, orig_time,
           error_site, error_db,
           log_time, sys_trans_id,
           app_usr)
select
    orig_site,
    orig_db,
    orig_user,
    orig_time,
    error_site,
    error_db,
    log_time,
    sys_trans_id,
    app_usr
from    rs_exceptshdr exh

/* add logged command counts to table */

update #tab1
  set reccount = (select max(src_cmd_line)
                    from   rs_exceptscmd exc
                      where exc.sys_trans_id = #tab1.sys_trans_id
                    group by exc.sys_trans_id)
/* print summary */

begin
  select @cnt = count(sys_trans_id)
  from rs_exceptshdr

  select @ccnt = convert(char(8), @cnt)

  if (@cnt = 0)
  begin
    print " "
    /* 20500,"         There are 0 Logged Transactions." */
    exec rs_get_msg 20500, @msg output
    print @msg
    print " "
    return
  end

  print " "
  /* 20501,"         Summary of Logged Transactions on ‘%1!’", @rsname
*/
  exec rs_get_msg 20501, @msg output
  print @msg, @rsname
  print " "
  print " "

  if (convert(int, 0×0000100) = 65536)
  begin
    select
      "Xact ID" = convert(int, reverse(substring(sys_trans_id, 5, 8))),
      "Org Site" = substring(rtrim(orig_site)+"."+rtrim(orig_db), 1, 15),
      "Org User" = substring(orig_user, 1, 8),
      "Org Date " = convert(char(11), orig_time),
      "Dest Site" = substring(rtrim(error_site)+"."+rtrim(error_db), 1,
15),
      "# Recs/Xact" = reccount
    from #tab1
  end
  else
  begin
    select
      "Xact ID" = convert(int, substring(sys_trans_id, 5, 8)),
      "Org Site" = substring(rtrim(orig_site)+"."+rtrim(orig_db), 1, 15),
      "Org User" = substring(orig_user, 1, 8),
      "Org Date " = convert(char(11), orig_time),
      "Dest Site" = substring(rtrim(error_site)+"."+rtrim(error_db), 1,
15),
      "# Recs/Xact" = reccount
    from #tab1
  end

  print " "
  print " "
  /* 20502,"         To Delete a Specific Logged Xact., type
‘rs_delexception {Xact ID}’" */
  exec rs_get_msg 20502, @msg output
  print @msg
  print " "
end

select @row_cnt = 1

while (@row_cnt > 0)
begin

set rowcount 1

  select @systran = sys_trans_id from #tab1

  select @row_cnt = @@rowcount

  delete #tab1 where @systran = sys_trans_id

set rowcount 0

  if (@row_cnt = 0)
  begin
    print " "
    print "All exceptions deleted"
    print " "
    return
  end

  if (convert(int, 0×0000100) = 65536)
  begin
    select @cxact = convert(char(8), convert(int,
reverse(substring(@systran, 5, 8))))
  end
  else
  begin
    select @cxact =  convert(char(8), convert(int, substring(@systran, 5, 8)))
  end

  /* if logged transaction exists, delete it. */

  select @cmdcount = rtrim(convert(char(9), reccount))
  from   #tab1
  where  sys_trans_id = @systran

  print " "
  /* 20505," Deleting %1! Commands in Logged Transaction # %2! on ‘%3!’" */
  exec rs_get_msg 20505, @msg output
  print @msg, @cmdcount, @cxact, @rsname
  print " "
  print " "

  begin transaction

  delete  rs_systext
  from    rs_exceptscmd exc,
          rs_exceptshdr exh,
          rs_systext sys
  where   exc.sys_trans_id = exh.sys_trans_id
  and     exc.cmd_id = sys.parentid
  and     sys.texttype = "C"
  and     exh.sys_trans_id  =  @systran

  select @err = @@error

  if (@err != 0)
  begin
    select @tab_name = ‘rs_systext’
    /* 20506,"Deleting %1! table failed. Transaction Rolled Back." */
    exec rs_get_msg 20506, @msg output
    print @msg, @tab_name
    rollback transaction
  end
  else
  begin
    delete  rs_exceptscmd
    where   sys_trans_id  =  @systran

    select @err = @@error

    if (@err != 0)
    begin
      select @tab_name = ‘rs_exceptscmd’
      /* 20506,"Deleting %1! table failed. Transaction Rolled Back." */
      exec rs_get_msg 20506, @msg output
      print @msg, @tab_name
      rollback transaction
    end
    else
    begin
      delete  rs_exceptshdr
      where   sys_trans_id  =  @systran

      select @err = @@error

      if (@err != 0)
      begin
        select @tab_name = ‘rs_exceptshdr’
        /* 20506,"Deleting %1! table failed. Transaction Rolled Back." */
        exec rs_get_msg 20506, @msg output
        print @msg, @tab_name
        rollback transaction
      end
      else
      begin
        commit transaction

        select @err = @@error

        if (@err !=0)
        begin
          /* 20509,"Executing ‘commit transaction’  failed. Transaction
Rolled Back." */
          exec rs_get_msg 20509, @msg output
          print @msg
          rollback transaction
        end
        else
        begin
          print " "
          /* 20510,"         Logged Transaction # %1! Successfully
Deleted. Truncate RSSD Transaction Log if Necessary." */
          exec rs_get_msg 20510, @msg output
          print @msg, @cxact
          print " "
        end
      end
    end
  end
end
go

EXEC sp_procxmode ‘dbo.rs_del_all_exception’,'unchained’
go

IF OBJECT_ID(‘dbo.rs_del_all_exception’) IS NOT NULL
    PRINT ‘< << CREATED PROCEDURE dbo.rs_del_all_exception >>>’
ELSE
    PRINT ‘< << FAILED CREATING PROCEDURE dbo.rs_del_all_exception >>>’
go

Share Button

Sybase Replication Server: ignoring duplicate keys

Sybase’s Replication Server allows you to replicate data entry from one database into another (there can be more than one replicate database).  They don’t necessarily have to be even from the same vendor.

Duplicate rows will occur when an application inserts data into the primary and replicate database(s), if the data being entered in a replicated table.  Replication Server’s DSI connection will stop saying that it has detected a duplicate key and requires a DBA to tell it what to do.  If this duplicate key can be ignored, then the DBA will skip the transaction, which will make a note of the transaction and will skip it (go on to the next transaction).

resume connection to MYSERVER.MYDB skip transaction

The problem with this approach is that if there are a lot of duplicate keys, not only could you be sitting for a while skipping the transactions, you run the risk of skipping a transaction that isn’t a duplicate key.  Say if someone deleted the table on the replicate database..  You could easily make a mess of things if you arbitrarily skip transactions.

Replication Server has a feature called error classes that you can define the course of action if an error occurs with a DSI connection.  The only real issue is that the lowest level of granularity is at the DSI connection level and the highest is all insert dbms type (i.e. ASE) replicated systems.  To create an error class:

create error class ASEallowdupsErrorClass

The error classes can be inherited so if you wanted an error class to ignore duplicate keys and another to stop replication on a duplicate key, you would do something like so:

RSSD> rs_init_erroractions ASEallowdupsErrorClass, rs_sqlserver_error_class

Sybase ASE’s error number for a duplicate key is 2601, but ASE will also raise the 3621 (aborted transaction) error.  We need to set the error class ASEallowdupsErrorClass to ignore duplicate keys:

assign action ignore for ASEallowdupsErrorClass to 2601
assign action ignore for ASEallowdupsErrorClass to 3621

Now that we’ve created the error class and set it to ignore duplicates, we need to do two last things:

  1. alter the DSI connections to use the new error class
  2. suspend and then resume the DSI connections for the DSIs to use the new error class
alter connection to MYSERVER.MYDB
set error class to ASEallowdupsErrorClass
go
suspend connection to MYSERVER.MYDB
go
resume connection to MYSERVER.MYDB
go

Generally, applications should not be performing data entry of the same data across the replicated databases as Replication Server is made for it.

Share Button