Jason L. Froebe: Ramblings of a Geek

Because Sybase has yet to address the GLIBC (LD_POINTER_GUARD) issue with any of their products on Linux, we are forced to find alternative ways of getting Sybase software to run on modern Linux distribution.  In this situation, we will be using the Ubuntu 8.04 Just Enough OS (Ubuntu JEOS) Linux distribution.

After installing Ubuntu 8.04 Server JEOS on the physical or virtual machine, download the Ubuntu 7.04 Server iso image.  Why Ubuntu 7.04?  Ubuntu 7.04 is the last Ubuntu release that has an old enough GLIBC that Sybase software will work with.

% sudo debootstrap feisty /home/sybase file:///media/cdrom0

• Change the home directory of the sybase user to be /dbms/sybase by editing the /etc/passwd file

% sudo cp /etc/resolv.conf /home/sybase/etc/resolv.conf

% sudo cp /etc/hosts /home/sybase/etc/hosts

% sudo cp /etc/passwd /home/sybase/etc/passwd

% sudo cp /etc/shadow /home/sybase/etc/shadow

% sudo `cat /etc/apt/sources.list | sed -e ’s/hardy/feisty/g’ > /home/sybase/etc/apt/sources.list`

% sudo mount –bind /dev /home/sybase/dev

% sudo mount –bind /proc /home/sybase/proc

% sudo mount -t devpts none /home/sybase/dev/pts

% sudo mount –bind /tmp /home/sybase/tmp

% sudo aptitude update

% sudo aptitude upgrade

% sudo apt-get install locales dialog wget debconf devscripts gnupg

% sudo aptitude install libstdc++5 libstdc++6 libaio

• If you’re installing v12.5x software, you will need to install an even older copy of libstdc++ (not needed for v15 or higher software):

% aptitude install libstdc++2.10-glibc2.2

• Install Sybase software into /dbms/sybase (as root in the chroot environment which you should still be in)

% sudo cp /dbms/sybase/SYBASE.sh /dbms/sybase/SYBASE.env

% sudo echo DSQUERY=MYASE >> /dbms/sybase/.SYBASE.env

% sudo echo sa_password > /dbms/sybase/.sapwd

% sudo chmod go-rwx /dbms/sybase/.sapwd

% sudo chmod u-wx /dbms/sybase/.sapwd

% sudo chown -R sybase /dbms/sybase

• Install the sybase_chroot script into /usr/local/bin and make it executable by root

• Install the sybase_ase rc script into /etc/init.d and make it executable by root

• Exit the chroot environment by typing ‘exit’

• create a symbolic link outside of the chroot environment to make /home/sybase/dbms appear as /dbms.  This will allow you to access the Sybase software, notably OpenClient, as a normal user outside of the chroot environment.

% sudo ln -s /home/sybase/dbms /dbms

You are now able to start Sybase ASE using sudo /etc/init.d/sybase_ase start and stop Sybase ASE using sudo /etc/init.d/sybase_ase stop.  Starting Sybase Replication Server, OpenServer, or similar Sybase software is simply a matter of copying the /etc/init.d/sybase_ase and tweaking the script copy.

If you feel this is way complicated and Sybase should just fix their software, let them know.  Please point them to this page and refer to CR455393.

UPDATE: The GLIBC issue is fixed in v15.0.2 esd 4.  It doesn’t seem to be working for everyone though.  More information is needed about those systems where esd 4 doesn’t resolve the matter.  No fix is available for 12.5x from Sybase.

UPDATE: Keep in mind, that the Sybase ASE 15.0.2 esd 4 patch is available only to Sybase customers with a current maintenance contract with Sybase.

Listen to this podcast

For those of us that use Sybase’s Replication Server, we have long ago been pacified into believing that there really isn’t much that can be done with Replication Server’s quirks but to endure them.

In today’s edition of the Sybase RepServer Quirks we take a look at the “out of mutexes” error.  A mutex is an exclusive lock on a resource.  If you receive an error about RepServer running out of mutexes, you just don’t have enough defined through RepServer.  Simple huh?

Of course it is.  Ahh..  but then how do how many mutexes your particular Sybase RepServer needs?  How many grains of sand are on the moon?  According to Sybase you should know this already but since you can’t read minds (I hope you can’t because I’m thinking of donuts and not RepServer), I’ll pass on what Sybase is saying when pressed about it (Greg Carter @ Sybase):

> Just to elaborate a bit more on this question of the number of mutexes; as
> you probably know the mutex requirements for RepServer increased
> dramatically with the SMP feature. I have since struggled to come up with a
> formula for estimating mutex requirements so that you may properly set the
> “num_mutexes” configuration. While in the latest iteration of this formula
> I have satisfied myself that all mutexes have been accounted for, still the
> estimate it provides seems to fall short in some cases.
>
> Recent investigations by Connectivity seem to indicate that the problem may
> not be with sizing mutex requirements, but rather with sizing message queue
> (”num_msgqueues”) requirements. It appears that Open Server may be using
> the total of the settings for “num_mutexes” and “num_msgqueues” as the
> upper bound for the creation of these two objects together. So it may be
> that even though “num_mutexes” has been sized properly, if “num_msgqueues”
> is too low then you may see a message regarding the failure to create a
> mutex or the failure to create a message queue depending on which one was
> being created at the time that upper bound was surpassed.
>
> The moral here is that until Open Server resolves this issue you need to
> verify the sizing of both “num_mutexes” and “num_msgqueues” in the event
> that either error message appears since you can not rely on the message to
> indicate which one is low.
>
> For your convenience I’ll include here the latest formulas for estimating
> mutex and message queue requirements. Note that the one for mutexes may not
> agree completely with the one that is given in the 12.6 SMP White Paper - I
> have not compared them.
>
> Mutex requirements for the optimized binary:
> num_mutexes = 75 + Num(partitions) + 4*Num(DSI/S) + 3*Num(DSI/E) +
> 2*Num(Dist) + 2*Num(RepAgent Exec) + 2*Num(RSI User) + 5*Num(Queues) +
> 5*Num(SQT Cache) + Num(rs_subscriptions rows) + Num(RSSD tables) +
> Setting(cm_max_connections) +
> 2*MAX(Admin connections) + 2*Num(Other Connections) + Num(Origins) +
> 2*Num(Threads) + MAX(subscription (de)mat)
>
> Where
> - “Other Connections” are connection to this RepServer including ID Server
> connections, RSM connections, etc.
> - “Origins” are the different origins (or primary databases) that could
> possibly have transactions flowing through this RepServer, whether by a
> RepAgent or by a route (intermediate included)
> - “Threads” includes every thread RepServer may start. These are the
> “Global” thread, the “Initialization” thread, threads for each of the
> daemons (dAIO, dCM, dVersion, dRec, dSub, dStats, dAlarm), RepAgent User
> threads, SQM Writer threads, SQT threads, Distributor threads, DSI/S and
> DSI/E threads, RSI User and RSI threads.
> - “subscription (de)mat” is the number of asynchronous subscription
> management requests for materializing or dematerializing that may be taking
> place at any moment.
>
> Note: For the diagnostic binary you will need to double the figure
> determined with the above formula.
>
> Message queue requirements for the optimized and diagnostic binary:
> num_msgqueues = 10 + Num(DSI/S) + Num(DSI/E) + Num(Queues) + Num(Dist)
>
> Thanks,
> G.Carter

Neither this explanation or the two equations are anywhere in the manuals.  I’ve opened feature request 485482 for RepServer to handle this automagically as there really is NO NEED for a RepServer admin to have to worry about this.  If you are or ever have run into this problem, give Sybase a holler and tell them to fix this bug.

Listen to this podcast

There is a replay of the Jeff Talman’s RepServer Parallel DSI presentation hosted by ISUG from October 3rd, 2007 at:

http://my.isug.com/p/do/sd/sid=1035&type=0

This is available only to current ISUG members, so if you’re not a member, please join ISUG now

Listen to this podcast

Hello,
Here’s the ISUG connection information for Jeff Tallman’s presentation
[today, October 3rd, at 11am CDT]:
http://my.isug.com/index.php?mo=fo&op=st&thread=5406

FIRSTTIME USERS: To save time before the meeting, check your system
to make sure it is compatible with Microsoft Office Live Meeting:
http://go.microsoft.com/fwlink/?linkid=52354
For assistance, visit Live Meeting Help and Support:

http://r.office.microsoft.com/r/rlidLiveMeeting?p1=7&p2=en_US&p3=LMInfo&p4=support—-We hope to have streaming audio available for this presentation for
those
that just want to listen and not ask questions at the end.

If there are any questions, please don’t hesitate to ask.

Regards,

Mike Harrold
Executive Director
International Sybase User Group
Email: Mike.Harrold@isug.com
Support the Sybase Community and Join ISUG today at http://my.isug.com/join

Listen to this podcast

Hi everyone,

I just wanted to take a couple of minutes of your time and ask you for
feedback about ISUG. Please take a moment to complete the ISUG
Survey at:

http://my.isug. com/index. php?module= survey&op= respond&sid= 20

More than 80% of the people subscribed to this mailing list are not
members of ISUG. We are very interested in hearing from those
people, especially as to why they are not members. So, if you are
included in this group, please do take a moment to share your thoughts
with us.

As a special thank you, upon completion of the survey you’ll be given
a discount code worth $10 off a new membership (or a membership
renewal) for the Silver and Gold benefit packages!

You will be prompted to login when you click on the survey link,
unless you are currently logged in. All subscribers to this mailing
list already have accounts. If you have never logged in to My.ISUG.com
before, you can request a password reset at:
http://my.isug. com/loginsupport .php — simply enter the email
address where you have received this message and a temporary password
will be sent to you.

Thank you everyone. We look forward to reading your responses.

Regards,

Mike Harrold
Executive Director
International Sybase User Group
Email: Mike.Harrold@ isug.com
Support the Sybase Community and
Join ISUG today at http://my.isug. com/join

Mike Harrold from ISUG posted the above email to the sybase-l mailing list.  Please let ISUG know why you are or aren’t an ISUG member.

thanks

Jason

Listen to this podcast

Sybase announced today that the end of life of the Portal Suite will occur on August 16, 2008. Normally end of life’s occur at the end of the month so I’m curious as to why the 16th was chosen.

To be honest, I never really liked the concept of a web portal as they seemed to be very dependent on java applets and active-x… many times they would either crash your browser or be very insecure as much of the traffic was unencrypted. Sybase’s implementation was better than most.

Portals have for the most part been replaced by AJAX enabled sites do more and require less of the browser. Granted, there are some real security issues with AJAX implementations (AJAX frameworks) that have yet to be addressed, so while the rush to convert to an AJAX powered web 2.0 site is occurring, I hope people do their best to secure them. Enough rambling.

Read the rest of this entry »

Listen to this podcast

When you try to install a Sybase patch for Openclient, ASE, ASIQ or a myriad of other Sybase products, you may receive an error such as:

“ASE 12.5.4 is NOT installed”

This can appear if you delete the $SYBASE/installed and/or the $SYBASE/uninstall directories. (Why would you delete those directories? They are not needed for the operation of any Sybase product, therefore can be deleted for space reasons.)

So how to install the patch? Set INSTALL_ALL_PATCH to “1″ prior to running setup:

export INSTALL_ALL_PATCH=1

Listen to this podcast

The class registration and listing is available on Sybase’s TechWave website

Here is my current schedule:

My Agenda     Time Session Title Learning Track Status Tuesday, August 7, 2007 2:30 pm - 5:00 pm  EDU106-1 Performance and Tuning: ASE Query Optimization   Education Courses and iAnywhere TechSummits Registered Wednesday, August 8, 2007 8:00 am - 10:15 am  EDU106-2 Performance and Tuning: ASE Query Optimization   Education Courses and iAnywhere TechSummits Registered 1:00 pm - 2:00 pm  DAT508 Using Java and Web Services within Adaptive Server Enterprise (ASE)   Enterprise Database, Data Management and Business Continuity Registered 2:15 pm - 3:45 pm  DEV311 Designing High Performance Applications/Transaction Design (Prevention not Diagnosis & Medication)   Application Development Trends Registered 4:00 pm - 5:30 pm  DEV339 Case-Express: Using Sybase tools and bleeding edge technologies for a feature-rich web application   Application Development Trends Registered Thursday, August 9, 2007 8:00 am - 10:15 am  EDU114-3 Migrating from ASE to Sybase IQ: An ASE DBA’s perspective   Education Courses and iAnywhere TechSummits Registered 1:00 pm - 2:00 pm  IAW410 Setting up Replication Server and OpenSwitch for Continuous Availability and Disaster Recovery   Data Integration, Analysis, and Warehousing Registered 3:30 pm - 5:00 pm  DEV319 Connecting Adobe Flex and Flash to Sybase Adaptive Server Enterprise (ASE) using Web Services   Application Development Trends Registered Friday, August 10, 2007 8:00 am - 10:15 am  EDU106-4 Performance and Tuning: ASE Query Optimization   Education Courses and iAnywhere TechSummits Registered 1:00 pm - 2:30 pm  DAT502B Top Ten Adaptive Server Enterprise (ASE) MDA Tables   Enterprise Database, Data Management and Business Continuity Registered

Listen to this podcast

As you may remember, my good friend Ken Rearick created an excellent stored procedure, rs_del_all_exception, that safely clears out old exceptions from the RSSD database. Ken has just sent an updated version to me. B-)

Download: rs_del_exception stored procedure

Listen to this podcast

Remember my Sybase Replication Server: custom built RepAgents with Perl! post back last year? Well, not only am I the new maintainer, but I finally have it working in a test environment. It isn’t ready for a new release for Sybase::TdsServer and Sybase::RepAgent, yet, but expect one soon.

My main development environment for these two modules is:

• Windows XP SP2
• ActiveState Perl 5.8.8
• Sybase Workspace 1.7 (Eclipse based)
1. EPIC Perl plugin
2. Subclipse (subversion client)

I’m hoping to get out my first CPAN release to the perl testers within the next month or so. Why the delay? My wife and I are expecting family and friends to stay with us for the next few weeks. That’s a very good thing

Listen to this podcast

Back at the dawn of the new millennia, Mark A. Parsons created and presented Replicating Logins, Passwords, Users, and Groups at the Sybase TechWave Australasia 2000 conference.  He provides different methods on how to keep specific tables in the master databases in sync across replicated servers.  This is extremely important in warm standby environments.

Kudos to Mark!

Mark Parsons is an independent consultant currently working in North America. He has been working with Sybase ASE since 1991. His skill set covers all areas touching on the ASE dataserver, with recent emphasis on performance and tuning as well as production support and troubleshooting.Joined TeamSybase: 2006
Products Supported: Adaptive Server Enterprise, Replication Server

Listen to this podcast

The replication capabilities of MySQL are relatively primitive when compared to the more mature Replication Server from Sybase. Sybase expects to support full featured replication using Sybase’s Replication Server in late 2008 with MySQL. This would be an add-on (RepServer Options) to the standard Replication Server product. Keep in mind that this is a tentative time frame and it can arrive sooner or later than 2008.

Listen to this podcast

If we several large transactions, the sqt_cache fills up for the SQM of the warm standby, spits out a warning, and after a few days, this individual sqt_cache exceeds 180Mbytes and blows away the max memory (200MBytes).  RepServer should not be dying because of this.  In fact, repserver shouldn’t allocate more memory to this particular sqt_cache than the maximum of 30MBytes.

I see two bugs:

1. An individual sqt_cache is allocated more than the maximum sqt_cache configuration
2. RepServer crashes by overallocation of memory

I’ve opened a techsupport case with Sybase and will be let everyone know what the bug # will be and when we can expect it fixed.

SQMs: 3
Warm Standby set up for one database. Five parallel DSIs.
max_sqt_cache size: 30MBytes
dsi_max_sqt_cache_size for warm standby connection 30MBytes
dsi_max_sqt_cache_size for all other DSIs (for RSSD and back to primary of warm standby): 1MByte

I. 2007/02/28 03:33:21. Server using Open Server version Sybase Server-Library/12.5.1/P-EBF12506-12505 ESD #8/DRV.12.5.1.3/SPARC/Solaris 2.8 Native Threads/BUILD1251-041/OPT/Sun Apr 10 21:02:15 2005

I. 2007/02/28 03:33:21. Server using Open Client version Sybase Client-Library/12.5.1/P-EBF12505 ESD #8/DRV.12.5.1.3/SPARC/Solaris 2.8 Native Threads/BUILD1251-041/OPT/Sun Apr 10 21:00:10 2005

….

W. 2007/02/28 12:28:39. WARNING #24057 DSI EXEC(105(1) sun_ase1.my_db) - t/sqtint.c(6219)_sqt_remove_largest_tran(102:1 sun_ase1.my_db): No candidate found for removal. Memory limit will be exceeded by SQM/TI thread.….T. 2007/03/03 23:40:58. (34): Additional allocation of 8176 bytes to the currently allocated memory of 209711456 bytes would exceed the memory_limit of 209715200 specified in the configuration.F. 2007/03/03 23:40:58. FATAL ERROR #7035 REP AGENT(sun_ase1.my_db) - /packapp.c(79)Additional allocation would exceed the memory_limit of ‘209715200′ specified in the configuration.T. 2007/03/03 23:40:58. (34): Exiting due to a fatal error

Listen to this podcast

It is more dangerous to run RepServer with the RSSD residing on ASE.  Why?  If the ASE RSSD goes down unexpectedly, RepServer doesn’t immediately know this, when it does, it dies … HARD.  Restarting both ASE and RepServer, you might discover that your stable queues are partially scrambled or missing data.

This was brought to the attention of RepServer Engineering almost a decade ago (by a certain fellow named of Zhong) but no fix has been made… "it is not a bug".

The ONLY advantage of running RepServer with the RSSD on ASE is if you have multiple RepServers on the same box and you wish to save on a bit of administration.  Basically it is the ease of use.

So, put your RSSD in ASE if you must, but you’re probably better off with the embedded RSSD.

Listen to this podcast

The old exceptions consume space and can cause slow response of the rs_helpexception stored procedure along with other performance issues.  I’ve create a simple stored procedure (sans error checking) that will retain only the past X days of exceptions.  By default, we retain 7 days.

Why no error checking?  Well, to properly report the error, you will want to have it in the format that your management requires.  (That may in fact be you

A word of warning:

The stored procedure will perform dump transaction with truncate_only after every 20,000 rows deleted and also after each table.  If you retain your transaction logs, you will need to modify the dump tran to dump to a file / tape as needed. 

Read the rest of this entry »

Listen to this podcast

I originally posted this as a reply to Nicolaw on Livejournal: 

After a great deal of inspection, Nagios won’t work for what I need but I’m taking specific design ideas from it. Not that the design of Nagios is all that novel, but some parts of it are well thought out while other parts you just want to take the CLUE BAT (TM) to the developers. One of the big problems with Nagios, and similarily Big Brother, is that they don’t scale very well at all without significant customization.

I’m in the design phase of monitoring only about 100 Sybase ASE and Replication servers so while we aren’t talking about a lot of servers, the amount of information we are going to gather relating to performance, space utilization, diagnostics, and maintenance is considerable. Since this is a project for work, it is unlikely to see the light of the outside world Maybe I’ll be able to convince the Powers that Be that releasing would be good PR. maybe..

Maybe on my spare time, I’ll create a lightweight Nagios plugin for the Sybase ASE and Sybase Replication server.

Oh, found out the ieee1394 driver in 2.6.17 doesn’t like a flakey external ieee1394 (firewire) harddrive too well. It causes my machine to restart. The enclosure also has a usb2 connection so I’ll try that this weekend. My other external harddrive (same model enclosure and hd model) works fine using ieee1394. I prefer ieee1394, because it is:

1. faster than usb2
2. less load on my box
3. I use my firewire card for something other than the Dazzle Hollywood Bridge

Update:

One of my biggest gripes about Nagios is that it doesn’t really have a repository (back end database) where the data can be mined. I’m sure I can add a repository for all the data but I’m unsure whether it is worth the effort if I’m not able to release the code back to the Nagios people.

Listen to this podcast

My good friend Ken Rearick wrote a rather clever little stored procedure for managing all the exceptions that are raised in the life of a Sybase Replication Server.  He posted this on sybase.public.rep-server a few days ago:

This is a rewriten rs_delexception script that will list out all exceptions and then step through deleting all of them. As this uses the same procedure for deleting the entries it does not cause any problems with integerity in the RSSD — Ken Rearick

Read the rest of this entry »

Listen to this podcast

This is just one way to resync the replicated database in Sybase
Read the rest of this entry »

Listen to this podcast

Sybase’s Replication Server allows you to replicate data entry from one database into another (there can be more than one replicate database).  They don’t necessarily have to be even from the same vendor.

Duplicate rows will occur when an application inserts data into the primary and replicate database(s), if the data being entered in a replicated table.  Replication Server’s DSI connection will stop saying that it has detected a duplicate key and requires a DBA to tell it what to do.  If this duplicate key can be ignored, then the DBA will skip the transaction, which will make a note of the transaction and will skip it (go on to the next transaction).

1: REP_SERVER> resume connection to MYSERVER.MYDB skip transaction

2: REP_SERVER> go

The problem with this approach is that if there are a lot of duplicate keys, not only could you be sitting for a while skipping the transactions, you run the risk of skipping a transaction that isn’t a duplicate key.  Say if someone deleted the table on the replicate database..  You could easily make a mess of things if you arbitrarily skip transactions.

Replication Server has a feature called error classes that you can define the course of action if an error occurs with a DSI connection.  The only real issue is that the lowest level of granularity is at the DSI connection level and the highest is all insert dbms type (i.e.