YAPC::NA VIM as a Perl IDE‎ – Damian Conway (2016)

Watch it. You will be impressed

Video by YAPC::NA 2016

Share Button

Video: From YAPC::2012 Tomas Doran Messaging, interoperability and log aggregation a new framework

Wouldn’t it be great to have a simple way of shipping logs, outputs, and so on to a central location with the option of pre-processing the data? Here is a good start. (There were technical issues in the first few minutes of the video)

Good job 🙂

Share Button

Getting Flickr::API to work on Ubuntu Linux SOLVED

In order to use the Flickr::API example by Gabor Szabo, I needed to create a LWP object specifying the path to the system certificates when creating the Flickr::API object.

Works:

my $flickr_api = Flickr::API->new( {
    key       => $key,
    # secret  => $secret,
    unicode   => 1,
    lwpobj    => LWP::UserAgent->new(
        ssl_opts => { 
            SSL_ca_path => '/etc/ssl/certs',
        },
    ),
}); 

Doesn’t work Perl(reports a 500 error with “Can’t connect to api.flickr.com:443 (certificate verify failed)” ) :

my $flickr_api = Flickr::API->new( {
    key       => $key,
    # secret  => $secret,
    unicode   => 1,
    ssl_opts  => { 
        SSL_ca_path       => '/etc/ssl/certs',
    },
});

You could use Mozilla::CA as brian d foy suggests but the bundled file is from 2014. Personally, I’ll use the certificate files from the OpenSSL project. 🙂

Share Button

Perl: How to check that the Perl Modules are installed at runtime

Often when you write a Perlprogram in Perl, you know which system(s) you’re going to be using the program on and you can ensure the required Perl modules are installed. Other times, you may not. Unless you want to be called simply because a required Perl module wasn’t installed on a system, you probably should check and provide a meaningful error message.

In this example, we’ll use the core module Module::Load in order to load a module. We also declare the %MISSING_MODULE hash:

use Module::Load;

our %MISSING_MODULE;

We populate the %noncore_modules with non-core modules we want to use. If the value for the module is an array, we will import those specific variables/subroutines:

BEGIN {
    # We're going to use:
    #   Pod::Usage with no special imports
    #   My::Secret::Module importing *ONLY* potato and carrot subroutines
    my %noncore_modules = (
        'Pod::Usage'         => 'default',
        'My::Secret::Module' => [ qw(
            potato
            carrot
            ) ],
    );

    foreach my $tmp_module (keys %noncore_modules ) { 
        if (eval { load $tmp_module ; 1; } ) { 
            if ( ref( $noncore_modules{$tmp_module} ) eq 'ARRAY' ) { 
                # Only specified subroutines/variables specified in
                #  %noncore_modules will be imported
                $tmp_module->import( @{ $noncore_modules{$tmp_module} } );
            } else {
                # anything in the @EXPORT of the module will be imported
                #  @EXPORT_OK will not be imported

                $tmp_module->import();
            }   
        } else {
            $MISSING_MODULE{$tmp_module} = 1;
        }   
    }
}

Now that we’ve loaded Perl modules that we could, let’s report on any that failed to load:

if (%MISSING_MODULE) {                                            
    $RC = 1;

    foreach my $perl_module (keys %MISSING_MODULE) {
        printf STDERR "Missing Perl module %s. Please have unix team install the module and verify permissions!\n", $perl_module;
        print STDERR "ABORTING\n";
    }

    exit $RC;
}
Share Button

Perl: Sourcing a profile or bashrc or other shell script SOLVED

Everyone has worked at Perla place where they do things slightly different than what you’re used to. In this case we need to source a shell script file that houses the environment variables that we need to import. Unfortunately, the shell script file may or may not call other scripts/programs or it may use string manipulation to populate the environment variables. This means you can’t just read a the file in perl with simple key/value pairings.

In the Unix/Linux shell scripting world, if you export an environment variable it will be available in any child process.

# Here we export the variable so it will show up in Perl's %ENV hash:
export MYVAR="woohoo"

If we don’t explicitly export the environment variable, it will not be available to a child process.

# We don't export the variable so it will not show up in Perl's %ENV hash:
NOTEXPORTED_VAR="too bad"

So how do we handle the non-exported environment variables so Perl can use them? Each shell that is POSIX compliant in one way or another will have the set builtin command that will produce output of the environment variables regardless of whether they’ve been exported. Fortunately for us, it is in key/value pairs with an equals sign “=” as the delimiter. Be warned, you will get everything.

In the example code below we’re going to use the BASH shell to source the /somedir/.env file. You can replace it with the shell of your choice. Setting an environment variable with Perl’s %ENV hash will automatically export it making it available for any child processes of the Perl process.

 BEGIN {
     # you will need to include the "&& set" *IF* you have an shell file
     #  that doesn't export the variables.  
     if ( -f '/somedir/.env' && -x '/somedir/.env') {
         open(my $PS, 'bash -c ". /somedir/.env && set" |') or die 'Cannot execute bash built-in set');

         while (< $PS>) {
             # we need to strip extended ASCII characters
             #  and any lines without an "="
             if (/=/ && /[^\x20-\x7F]/) {
                 chomp;
                 my ($key, $value) = split /=/;
                 $ENV{$key} = $value;
             }
         }

         close $PS;
     }
Share Button

The Perl Data Language Book: PDF fixed to work with tablet/phone PDF readers like Aldiko

Perl Data Language: Scientific computing with Perl

Aldiko Book Reader Premium Full v2.2.3.apkMany PDF readers for smart phones (Android/iphone) and tablets manage the pdf files Calibre_Logobased solely on the Title and Author fields in the PDF file. While for this is fine for your average book, it is not all that helpful with manuals that tend to have abbreviated or no data in the title/author fields.

How to fix? Easy. Go get Calibre. Drop the PDF files on to the running Calibre. Edit them by hitting the E key.

In my case, I edited the “Title”, “Author”, “Tags”, “Publisher” and “Languages”:

Calibre

Calibre doesn’t modify the PDF files themselves so I will need to export the files to a custom directory. In Calibre nomenclature, this is “Saving”. Highlight all the titles you want to export and hit “S” twice. Why twice? No idea. Choose the directory.

Perl Data Language Book for PDL 2.006

Perl Data Langauge

You can now copy the exported PDF files to your phone, tablet, whatever without fear of the v2.006 version of the PDL Book being rejected by Aldiko because the v2.004 version is already added.

Share Button

FW: A Faster Perl Runtime in Tiny Steps by Steffen Müller

Booking.com perluses the Perl programming language heavily across its entire technical infrastructure. At the size of our infrastructure, even small performance improvements can translate into hefty savings, not to mention the exciting new features that we will implement with those spare CPU cycles. For this reason, we are very happy to announce that we are now funding Dave Mitchell, prominently known for his many years of high-quality contributions to the Perl language implementation, to improve the Perl runtime’s performance one small step at a time.

Read more at A Faster Perl Runtime in Tiny Steps by Steffen Müller

Share Button

Perl: Error “local: not found” from PerlBrew when running older BASH versions or compatible shells (Korn / ksh) SOLVED

If you’re getting the “local: not found” error when running PerlBrew, we have the answer 🙂 The problem is that “local” doesn’t work in older versions of BASH or some other shells you might be using. You simply need to replace “local” with “typeset” in your ~/perl5/perlbrew/etc/bashrc file. Note that whenever you run “perlbrew init” or update the script, you will need to make the change again:

$ perlbrew use perl-5.8.9

A sub-shell is launched with perl-5.8.9 as the activated perl. Run 'exit' to finish it.

/home/wdc1212/.kshrc[13]: .[36]: local: not found [No such file or directory]
/home/wdc1212/.kshrc[13]: .[37]: local: not found [No such file or directory]
$ diff ~/perl5/perlbrew/etc/bashrc.old ~/perl5/perlbrew/etc/bashrc
36,37c36,37
< local code="$($perlbrew_command env $@)"
<     local exit_status="$?"
---
>     typeset code="$($perlbrew_command env $@)"
>     typeset exit_status="$?"
67,68c67,68
< local exit_status
<     local short_option
---
>     typeset exit_status
>     typeset short_option
$ perlbrew use perl-5.8.9<a href="http://perl.org"><img src="http://froebe.net/blog/wp-content/uploads/2010/02/perl-276x300.jpg" alt="perl" width="276" height="300" class="alignright size-medium wp-image-1207" /></a>
$ which perl
~/perl5/perlbrew/perls/perl-5.8.9/bin/perl
Share Button

HOWTO: Building Perl Module DBD::Sybase 1.15 for ActiveState Perl (Windows 32bit) 5.16.3 using the MinGW compiler

We no longer are tied to using Microsoft’s SAP SybaseVisual C++ compiler for building Michael Peppler’s Perl Module DBD::Sybase 1.15 for ActiveState Perl (Windows 32bit)!

We need to make a few changes to SAP Sybase’s Openclient though. Don’t worry, the changes are only needed to build the module. I used Sybase Openclient 15.7 ESD 7 but you should be able to use any 15.7 version of Openclient with minimal changes.

%SYBASE%\%SYBASE_OCS%\include\csconfig.h differences:

$ diff csconfig.h.old csconfig.h
58a59,63
> /* Load MinGW specific definitions */
> #if defined(__MINGW32__)
> #include "_mingw.h"
> #endif /* __MINGW32__ */
>
74c79
< #if ((SYB_MSC_VER >= 800)  || defined(__BORLANDC__))
---
> #if ((SYB_MSC_VER >= 800)  || defined(__BORLANDC__)) || defined(__MINGW32__)
84c89
< #else /* ((SYB_MSC_VER >= 800) || defined(__BORLANDC__)) */
---
> #else /* ((SYB_MSC_VER >= 800) || defined(__BORLANDC__)) || defined(__MINGW32__) */
94c99
< #endif /* ((SYB_MSC_VER >= 800) || defined(__BORLANDC__)) */
---
> #endif /* ((SYB_MSC_VER >= 800) || defined(__BORLANDC__)) || defined(__MINGW32__) */

%SYBASE%\%SYBASE_OCS%\include\sybfront.h differences:

$ diff sybfront.h.old sybfront.h
162c162
< #if !defined(_MSC_VER) && !defined(__BORLANDC__)
---
> #if !defined(_MSC_VER) && !defined(__BORLANDC__) && !defined(__MINGW32__)
180c180
< #if !defined(_MSC_VER) && !defined(__BORLANDC__)
---
> #if !defined(_MSC_VER) && !defined(__BORLANDC__) && !defined(__MINGW32__)

csconfig
sybfront

We now need to copy the *.lib files in %SYBASE%\%SYBASE_OCS%\lib appending “.a” in place of the “.lib” suffix:
libs

Don’t you perlthink SAP Sybase should add the changes above to OpenClient? I think so. Let your SAP Representative know!

Now we’re ready to build the module!

U:\build\DBD-Sybase-1.15>perl Makefile.PL
Set up gcc environment - 3.4.5 (mingw-vista special r3)
Sybase OpenClient 15.7 found.

By default DBD::Sybase 1.05 and later use the 'CHAINED' mode (where available)
when 'AutoCommit' is turned off. Versions 1.04 and older instead managed
the transactions explicitly with a 'BEGIN TRAN' before the first DML
statement. Using the 'CHAINED' mode is preferable as it is the way that
Sybase implements AutoCommit handling for both its ODBC and JDBC drivers.

Use 'CHAINED' mode by default (Y/N) [Y]:

The DBD::Sybase module need access to a Sybase server to run the tests.
To clear an entry please enter 'undef'
Sybase server to use (default: SYBASE): test_svr
User ID to log in to Sybase (default: sa):
Password (default: undef):
Sybase database to use on sd02 (default: undef): tempdb

* Writing login information, including password, to file PWD.

Checking if your kit is complete...
Looks good
Warning (mostly harmless): No library found for -llibsybtcl.lib
Warning (mostly harmless): No library found for -llibsybcomn.lib
Warning (mostly harmless): No library found for -llibsybintl.lib
Multiple copies of Driver.xst found in: C:/Perl/site/lib/auto/DBI/ C:/Perl/lib/auto/DBI/ at Makefile.PL line 80.
Using DBI 1.63 (for perl 5.016003 on MSWin32-x86-multi-thread) installed in C:/Perl/site/lib/auto/DBI/
Generating a dmake-style Makefile
Writing Makefile for DBD::Sybase
Writing MYMETA.yml and MYMETA.json

So far so good, so let’s continue with compiling and building the module:

U:\build\DBD-Sybase-1.15>dmake
dmake.exe:  makefile:  line 454:  Warning: -- Macro `BOOTSTRAP' redefined after use
cp dbd-sybase.pod blib\lib\DBD\dbd-sybase.pod
cp Sybase.pm blib\lib\DBD\Sybase.pm
C:\Perl\bin\perl.exe -p -e "s/~DRIVER~/Sybase/g" C:\Perl\site\lib\auto\DBI\Driver.xst > Sybase.xsi
C:\Perl\bin\perl.exe C:\Perl\site\lib\ExtUtils\xsubpp  -typemap C:\Perl\lib\ExtUtils\typemap  Sybase.xs > Sybase.xsc && C:\Perl\bin\perl.exe -MExtUtil
s::Command -e mv -- Sybase.xsc Sybase.c
C:\Perl\site\bin\gcc.exe -c  -IC:\Sybase/OCS-15_0/include -IC:/Perl/site/lib/auto/DBI   -DNDEBUG -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL_TEXTMODE_SCRIPT
S -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T -DHASATTRIBUTE -fno-strict-aliasing -mms-bitfields
-O2       -DVERSION=\"1.15\"    -DXS_VERSION=\"1.15\"  "-IC:\Perl\lib\CORE"   Sybase.c
C:\Perl\site\bin\gcc.exe -c  -IC:\Sybase/OCS-15_0/include -IC:/Perl/site/lib/auto/DBI   -DNDEBUG -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL_TEXTMODE_SCRIPT
S -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T -DHASATTRIBUTE -fno-strict-aliasing -mms-bitfields
-O2       -DVERSION=\"1.15\"    -DXS_VERSION=\"1.15\"  "-IC:\Perl\lib\CORE"   dbdimp.c
dbdimp.c: In function `syb_st_execute':
dbdimp.c:3906: warning: passing arg 2 of `ct_results' from incompatible pointer type
dbdimp.c:3930: warning: passing arg 5 of `ct_bind' from incompatible pointer type
Running Mkbootstrap for DBD::Sybase ()
C:\Perl\bin\perl.exe -MExtUtils::Command -e chmod -- 644 Sybase.bs
C:\Perl\bin\perl.exe -MExtUtils::Mksymlists \
     -e "Mksymlists('NAME'=>\"DBD::Sybase\", 'DLBASE' => 'Sybase', 'DL_FUNCS' => {  }, 'FUNCLIST' => [], 'IMPORTS' => {  }, 'DL_VARS' => []);"
Set up gcc environment - 3.4.5 (mingw-vista special r3)
C:\Perl\site\bin\dlltool.exe --def Sybase.def --output-exp dll.exp
C:\Perl\site\bin\g++.exe -o blib\arch\auto\DBD\Sybase\Sybase.dll -Wl,--base-file -Wl,dll.base -LC:\Sybase/OCS-15_0/lib -mdll -L"C:\Perl\lib\CORE" Syba
se.o    dbdimp.o   C:\Perl\lib\CORE\perl516.lib C:\Sybase\OCS-15_0\lib\libsybct.lib C:\Sybase\OCS-15_0\lib\libsybcs.lib C:\Sybase\OCS-15_0\lib\libsybb
lk.lib C:\Perl\site\lib\auto\MinGW\lib\libm.a C:\Perl\site\lib\auto\MinGW\lib\libkernel32.a C:\Perl\site\lib\auto\MinGW\lib\libuser32.a C:\Perl\site\l
ib\auto\MinGW\lib\libgdi32.a C:\Perl\site\lib\auto\MinGW\lib\libwinspool.a C:\Perl\site\lib\auto\MinGW\lib\libcomdlg32.a C:\Perl\site\lib\auto\MinGW\l
ib\libadvapi32.a C:\Perl\site\lib\auto\MinGW\lib\libshell32.a C:\Perl\site\lib\auto\MinGW\lib\libole32.a C:\Perl\site\lib\auto\MinGW\lib\liboleaut32.a
 C:\Perl\site\lib\auto\MinGW\lib\libnetapi32.a C:\Perl\site\lib\auto\MinGW\lib\libuuid.a C:\Perl\site\lib\auto\MinGW\lib\libws2_32.a C:\Perl\site\lib\
auto\MinGW\lib\libmpr.a C:\Perl\site\lib\auto\MinGW\lib\libwinmm.a C:\Perl\site\lib\auto\MinGW\lib\libversion.a C:\Perl\site\lib\auto\MinGW\lib\libodb
c32.a C:\Perl\site\lib\auto\MinGW\lib\libodbccp32.a C:\Perl\site\lib\auto\MinGW\lib\libcomctl32.a C:\Perl\site\lib\auto\MinGW\lib\libmsvcrt.a dll.exp
C:\Perl\site\bin\dlltool.exe --def Sybase.def --base-file dll.base --output-exp dll.exp
C:\Perl\site\bin\g++.exe -o blib\arch\auto\DBD\Sybase\Sybase.dll -LC:\Sybase/OCS-15_0/lib -mdll -L"C:\Perl\lib\CORE" Sybase.o   dbdimp.o   C:\Perl\lib
\CORE\perl516.lib C:\Sybase\OCS-15_0\lib\libsybct.lib C:\Sybase\OCS-15_0\lib\libsybcs.lib C:\Sybase\OCS-15_0\lib\libsybblk.lib C:\Perl\site\lib\auto\M
inGW\lib\libm.a C:\Perl\site\lib\auto\MinGW\lib\libkernel32.a C:\Perl\site\lib\auto\MinGW\lib\libuser32.a C:\Perl\site\lib\auto\MinGW\lib\libgdi32.a C
:\Perl\site\lib\auto\MinGW\lib\libwinspool.a C:\Perl\site\lib\auto\MinGW\lib\libcomdlg32.a C:\Perl\site\lib\auto\MinGW\lib\libadvapi32.a C:\Perl\site\
lib\auto\MinGW\lib\libshell32.a C:\Perl\site\lib\auto\MinGW\lib\libole32.a C:\Perl\site\lib\auto\MinGW\lib\liboleaut32.a C:\Perl\site\lib\auto\MinGW\l
ib\libnetapi32.a C:\Perl\site\lib\auto\MinGW\lib\libuuid.a C:\Perl\site\lib\auto\MinGW\lib\libws2_32.a C:\Perl\site\lib\auto\MinGW\lib\libmpr.a C:\Per
l\site\lib\auto\MinGW\lib\libwinmm.a C:\Perl\site\lib\auto\MinGW\lib\libversion.a C:\Perl\site\lib\auto\MinGW\lib\libodbc32.a C:\Perl\site\lib\auto\Mi
nGW\lib\libodbccp32.a C:\Perl\site\lib\auto\MinGW\lib\libcomctl32.a C:\Perl\site\lib\auto\MinGW\lib\libmsvcrt.a dll.exp
C:\Perl\bin\perl.exe -MExtUtils::Command -e chmod -- 755 blib\arch\auto\DBD\Sybase\Sybase.dll

Now, you might be thinking that since it compiled it should be good huh? Nah, we test things here! 😉

U:\build\DBD-Sybase-1.15>dmake test
dmake.exe:  makefile:  line 454:  Warning: -- Macro `BOOTSTRAP' redefined after use
C:\Perl\bin\perl.exe "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib\lib', 'blib\arch')" t/*.t

t/autocommit.t .. ok
t/base.t ........ ok
t/exec.t ........ ok
t/fail.t ........ ok
t/login.t ....... 1/6 DBD::Sybase::db ping failed: ct_cmd_alloc failed at t/login.t line 32.
t/login.t ....... ok
t/main.t ........ ok
t/multi_sth.t ... ok
t/nsql.t ........ ok
t/place.t ....... ok
t/thread.t ...... skipped: this DBD::Sybase not configured to support iThreads
t/utf8.t ........ ok
t/xblk.t ........ ok
t/xblob.t ....... ok
All tests successful.
Files=13, Tests=242, 17 wallclock secs ( 0.16 usr +  0.22 sys =  0.37 CPU)
Result: PASS

No major issues so we’re ready to build ActiveState’s PPD file and get it ready distribute:

U:\build\DBD-Sybase-1.15>dmake ppd
dmake.exe:  makefile:  line 454:  Warning: -- Macro `BOOTSTRAP' redefined after use

Zip the blib directory and name it “DBD-Sybase-1.15.zip”. Create a new directory named “MSWin32-x86-multi-thread-5.16” and copy the zip file into it.

You need to tell PPM where to find the zip file. The PPD file is simply an XML document, so we can just update the codebase:

<softpkg NAME="DBD-Sybase" VERSION="1.15">
    <abstract>DBI driver for Sybase datasources</abstract>
    <author>Michael Peppler (mpeppler@peppler.org)</author>
    <implementation>
        <architecture NAME="MSWin32-x86-multi-thread-5.16"></architecture>
        <codebase HREF="MSWin32-x86-multi-thread-5.16\DBD-Sybase-1.15.zip"></codebase>
    </implementation>
</softpkg>

Let’s test that it actually installs using ActiveState’s PPM:

U:\build\DBD-Sybase-1.15>ppm install DBD-Sybase.ppd
Unpacking DBD-Sybase-1.15...done
Generating HTML for DBD-Sybase-1.15...done
Updating files in site area...done
   5 files installed

U:\build\DBD-Sybase-1.15>

We only need to zip up the PPD and the MSWin32-x86-multi-thread-5.16 directory listed in the PPD. It is now ready to distribute. How you do it is entirely up to you.
DBD-Sybase-1.15 for ActiveState Perl 5.16 Win32

For ActiveState Perl 64bit, you will need to use Microsoft Visual Studio. However, since the support of 64bit Perl on Windows by ActiveState is rather poor, I would recommend using the 32bit version instead.

Share Button

Perl Regular Expression Cheatsheet

Regular Expressions (V1)

regular_expressions_cheat_sheet

Regular Expressions (V2)

Regular Expressions Cheat Sheet (V2)The second version of the Regular Expressions Cheat Sheet, a quick reference guide for regular expressions, including symbols, ranges, grouping, assertions and some sample patterns to get you started. (The updated version is only $2.00 at http://www.addedbytes.com/cheat-sheets/)
Share Button