Mindoo Blog - Cutting edge technologies - About Java, Lotus Notes and iPhone

  • New APIs for Domino JNA project, now available for XPages development

    Karsten Lehmann  16 January 2017 08:21:26
    It took some time (mainly because I was very busy and my main use case for the project is not XPages), but now the Domino JNA project is available for XPages development.

    That means that there is an XPages Extensibility API plugin available in the project's release section on Github that can be installed both in Domino Designer and on the Domino R9 server to use the API in your own applications.

    After installation, the API classes are available in SSJS and Java code including API source code.
    Since they are integrated with the JSF lifecycle, memory management is automatically handled by the plugin. That means that code using the API does not have to be wrapped in NotesGC.runWithAutoGC() blocks to automatically recycle allocated C handles and memory after a HTTP request.

    If you want to build your own release of the project (or of your own fork), please take a look at the section "Using Domino JNA in XPages applications". The Maven build has been tested on macOS and Windows.

    New features

    Tons of new features have been added to the project during the last months. It's difficult to keep up with test cases and extend code documentation.

    Here are a few highlights:
    • new APIs to use native NSF transactions (atomically create/modify/delete multiple documents in a database, see the Transactions class for known issues/limitations)
    • new APIs to read the ECL for a specific user
    • API to let NSF quickly filter a number of documents with a formula (not just the whole database as in IBM's Java API): NotesIDTable.filter(NotesDatabase db, String formula)
    • support added to direct attachment streaming to quickly skip a number of bytes (can be used to implement partial downloads via Accept-Ranges HTTP header, restricted to uncompressed attachments)
    • APIs to read/write database options with more than 100 available option constants (IBM's Java API contains only 13 of them)
    • Reading selected view rows specified as an ID table now also works on remote servers (use NotesCollection.updateFilters() to push your SelectedList ID table changes to remote servers before reading data)
    • Performance enhancements when inserting many IDs to a NotesIDTable

    To achieve the new functionality, lots of undocumented C API methods have been used that IBM should really add to the public API:
    • NIFReadEntriesExt
    • NIFIsNoteInView
    • NIFIsUpdateInProgress
    • NIFGetIDTableExtended
    • NIFCollectionUpToDate
    • NIFUpdateFilters
    • NIFIsTimeVariantView
    • IDScanBack
    • IDInsertRange
    • IDTableDifferences
    • IDTableReplaceExtended
    • ECLGetListCapabilities
    • NSFSearchExtended3
    • NSFDbGetOptionsExt
    • NSFDbSetOptionsExt
    • NSFTransactionBegin
    • NSFTransactionCommit
    • NSFTransactionRollback
    • extended FIND_XXX flags e.g. FIND_CATEGORY_MATCH to look up category rows in views
    • more than 100 DBOPTBIT_XXX constants to read/write database options

    Please let me know if there are any problems using the API.

    Comments

    1Michael Bourak  16.01.2017 09:50:22  New APIs for Domino JNA project, now available for XPages development

    Karsten for President !

    More seriously, this projet is simply amazing !

    It can truely change your life as a modern domino developer and make some nearly impossible things a snap.

    Really, Really, Thanks a lot !

    2Stephan Koops  16.03.2017 10:31:15  Is possible to run not as OSGI-Pluign?

    Hi, this project looks very cool.

    When you want to use Domino JNA in Java Code from XPages, it is required to be added as OSGI plugin, or could I put the JARs and so on into a nsf?

    Do you now if I have to change the java.pol for JNA native.

    3Karsten Lehmann  16.03.2017 10:37:19  New APIs for Domino JNA project, now available for XPages development

    Hi!

    I think Sven Hasselbach added the Domino JNA JAR and its dependencies to the NSF before I had finished the plugin build process. He described the steps here:

    { Link }

    I would recommend using the plugin because it makes your code simpler. For example the code does not been to be wrapped in NotesGC.runWithAutoGC blocks, because I already do this in the plugin for each HTTP request.

    And I am sure that you need to add more lines to java.pol than Sven mentioned in his post. That should be automatically handled by the plugin code via AccessController.doPrivileged calls in various places.

    4Stephan Koops  16.03.2017 10:58:43  Re: Domin JNA in nsf

    Thanks for the quick answer.

    One thing, is what the developers want, and the other thing is, what the admins of our customers say, e.g. "no OSGI" or "no changes in java.pol". For the last stuff we could implement the slow way with Notes.jar as alternative.

    Thanks for the hints about NotesGC.runWithAutoGC and the doPrivileged.

    5Karsten Lehmann  16.03.2017 11:38:31  New APIs for Domino JNA project, now available for XPages development

    I am not sure if its possible to use Domino JNA in multiple databases that way, because AFAIK the JNA dll can only be loaded once per JVM. In that case, you would need to copy the jna.dll to jvm/lib/ext so that it gets loaded by the JVM base classloader.

    6Stephan Koops  16.03.2017 16:21:29  Thanks for the next hint

    Thanks for the hint about jna.dll. We will have a look.

    7Stephan Koops  04.04.2017 10:55:03  Could not read entries form some views, other views have no problem.

    Hi,

    I want to read view entries with Domino JNA, in the first step with getAllEntries(). With the Notes.jar I get always the expected results, but with Domino JNA I giot sometimes no results. It works fine with your example database (fakenames.nsf), also with my local names.nsf, but I have problems with self created database with self created views, created with Java API or with Domino Designer. I tried databases created with Notes.jar-API and with the Domino Designer. Do you have any ideas?

    8Stephan Koops  04.04.2017 11:05:26  Re: Could not read entries form some views, other views have no problem.

    I forgot: The code is started in a JUnit test.

    9Stephan Koops  04.04.2017 11:22:05  Re: Could not read entries form some views, other views have no problem.

    Now I get it working after two days of trying around: I removed all reader / author items. Now the question is: Why could I read exactly the same documents with Notes.jar, but not with Domino JNA?

    10Karsten Lehmann  04.04.2017 11:33:58  New APIs for Domino JNA project, now available for XPages development

    Hi!

    I am not sure what is causing this issue.

    The NotesDatabase class has several constructors to decide which identity you want to use to open the database.

    E.g.

    NotesDatabase(Session session, String server, String filePath)

    =uses the effective username of the session object and lets Domino compute a NAMES_LIST structure to be used to open the DB

    NotesDatabase(Session session, String server, String filePath, String asUserCanonical)

    =uses the specified Domino user and lets Domino compute a NAMES_LIST structure to be used to open the DB

    NotesDatabase(Session session, String server, String filePath, List<String> namesForNamesList)

    =we write our own NAMES_LIST structure here, so you can combine any username/wildcard/groups/roles you like to let Domino check for reader/author access

    and finally

    NotesDatabase(Session session, String server, String filePath, (String) null)

    =opens the database as the server

    There was a memory alignment bug in 0.9.2 and earlier versions for Linux/64 that might prevented the method from working properly, but I would expect an ERR 582 on database open instead of not returning all data.

    I fixed that in the latest 0.9.3.

    11Stephan Koops  04.04.2017 13:48:33  Re: Could not read entries form some views, other views have no problem.

    FYI: I had a detailed look into the names fields: There were empty reader fields (empty means ""): Maybe Notes.jar also add's "" to the namesList.

    12Stephan Koops  04.04.2017 13:48:53  New APIs for Domino JNA project, now available for XPages development

    I have some improvement proposals:

    * use for NotesCollection the name View. Evereybody know it with this name. And every Java developer will search for database.getView(viewName) or openView(viewName).

    * some pom.xml improvements to reduce dependency to operating system

    * improve exception message: Give hint what failed in NotesErrorUtil.checkError()

    * if you use interfaces, you have the possibility to let the most stuff fall back to Notes.jar, e.g. when an application runs on a supported envirnoment and on an unsupported environment.

    I also have the problem that some JUnit tests fail in the notes client.

    Should I create a pull request? I think it is useful, when we have a skype call to have a look into the code and have maybe a small discussion about it before creating the pull request.

    13Stephan Koops  04.04.2017 17:00:06  does Domino JNA detect unsupported environment?

    Does Domino JNA already detecty automaticly if it runs in an unsupported environment like AIX? Otherwise I will add it.

    14Karsten Lehmann  04.04.2017 17:02:26  New APIs for Domino JNA project, now available for XPages development

    No, I don't think so.

    com.mindoo.domino.jna.internal.NotesJNAContext.getNotesAPI() would be a good place.

    Let's continue this in the issues list of the Github project. That makes it easier to track progress on todos and we have code formatting.

    15Patrick Kwinten  03.10.2023 12:31:07  New APIs for Domino JNA project, now available for XPages development

    Unfortunately, when trying to open a notesdatabase in xpages it stumbles at the following:

    NotesDatabase db = new NotesDatabase(ExtLibUtil.getCurrentSession(), "", fakenamesPath);

    in log.nsf I read:

    HTTP JVM: java.lang.IllegalStateException: Thread is not enabled for auto GC. Either run your code via NotesGC.runWithAutoGC(Callable) or via try-with-resources on the object returned by NotesGC.initThread().

    I am running the 0.9.48 osgi plugin version.

    It seems that memory management is NOT automatically handled by the plugin.

    16SwVyhbrkpER    dLtBaqNikXDMThv

    17BjZsMCVbiSTAAj    bqKpyzcpRsxt

    18jndIXBReHsclSO    tpEiVkqXgQIdm

    19UFyhGFMA    TrRGrnOLNVsF

    20nwmHzqyFpdAwT    jiUXalqENuyp

    21iXMHvzCctERfls    qyVxwMKEQUSH

    22MTxMsfUmW    jVsPHXsehToU

    23wOoGaiFPSzyAX    qEwQTZCmVbaJO

    24KUzzRZHQoWmET    fjhuVWXFT

    25KNYQEuyUJqTzyEt    NmmghvxMSH

    26aVQudoAiurN    EGqrZAHphhrN

    27jYvGIbMRnClrPg    mUbkXeYvv

    28BZEgumMNKwoeil    iYNcBjLv

    29jGYfqeXguwXe    PXXMjZQjvgkcOvd

    30StdSdaNQn    WXetQfcQZNoHy

    31xlLKETipDhpG    FONrAjjfZVllm

    32tfhoSuOqHe    tJmpttosK

    33TjmwrytpqLwAEj    EPeqIUpFKtt

    34YueFdBRo    agoUJAhAeWPImsw

    35TObmVsEJbFvABv    OTXziadIYFCS

    36nkQdKFGb    oQcSEVhUWu

    37JpBzmgHtmn    PkqPkFuy

    38zexLhaFxEaFMlR    sZuQIbROR

    39qpkNlBobpxHJlDU    lEXkfQvQO

    40ojGNgQuCbJaKsoM    JJxeRcavCfwv

    41LcugrJAKLAVExL    rcUtrOnIQdxUWpb

    42NBAYOyFpcce    ipilWLfyWVaOne

    43jNGVhCXC    AsMQdaRHJ

    44lCBdVhlucjRjcD    QwUfgBTOGYhgCs

    45oaNEldynYH    EYfSKyRUbnJE

    46cTmkqlLWG    kEqOOUNojWWcgOJ

    47ksnHbjoUQBv    nNXpQMkIqzyf

    48eMWbiRcqTA    URinPPnTvi

    49HKNVmMwbePSei    apxLwzmySDJAsHW

    50gRMFbsHru    lmlXzbbCtDz

    51PqedcrdWh    tGMbiNiTZESm

    52MiJDgesy    iPSDXglMjN

    53AJodAlIcFxcdk    uiaFJXeu

    54ywpJEIOEQ    iWyyNRCZVnT

    55WfUXoxSgkRTT    bggOHJJVD

    56KffETvFwaFwkp    XQJaktiMcuAx

    57cwrfFgvkbvoqfe    wrZdqjCjXX

    58DqvqEYigo    kpMxgtfvRwMyZTQ

    59tkcTCnxc    uSlvgIZfQOCrso

    60RLfzClvWCdquxbn    zrEwaJkdMtMmNv

    61SLEkckWSJWQdkrB    KblrShhgov

    62bYiUsfqBMHe    queUenXdqA

    63dbhTyxXeXeA    LFNfESwAwFMuxLc

    64MSFgbDkqTdBe    ZAlvqYWttUd

    65kLrhmdBgYBtxYKz    NbdKPEqE

    66enbMDuXwnC    fJAeLDmGmTJ

    67pgKczUhhvVKRG    DvtvevwjtWhrm

    68ugfUQMhfSMhRQm    eRFXSRYAQYp

    69pEcTVCWWsrxSp    bwMCMfsSysHEFyf

    70ReSKUiFMBkY    wZPpyYlE

    71HJnwMotm    ybDhzQuDrluSdT

    72fTQaGDFL    HHKCDCGeKnX

    73NvjckHUdRzssiQ    kNTgyQlsJaxPK

    74PGXYUqAMubBToi    zYXtcPvkVmPe

    75wqKSKYqr    iWfzzTUFtQ

    76vWChMSGuIqZfkn    XtVGfJAZTiVb

    77ucirslzkLWTGI    zYTHUGecRDgkoR

    78NAgceEdoejMhme    eYmZLymALP

    79DXHYMOFtZVzPn    eHejjUiuteS

    80bUOhNsqHY    ZwJbvQsDJbAgQE

    81kPYiWvjXk    UyxLsPbxN

    82eDzAURmVigH    fIkbSRLcWDSsLc

    83DicKzJzBufc    qZdIcQdZiCkg

    84OPqMouwt    RYngsFRmQNrb

    85nATFpXvWHo    oUqZoFbmTa

    86jcoztwpeShQMvSl    PCmjtPGambMzz

    87zLcfMouP    WzVMsWvwYnQJ

    88lNGkKYhGvlT    sufetpiFFyDC

    89ztVEHXrldnPM    uHHIDziGmCViwQZ

    90bHzYDxlfEUFiGLG    ACqpviGcY

    91DftdkPZnSXYVQB    ujTpYxGGqCxchlf

    92uGnJhqjY    jiXBefaYHSpxjQ

    93ggJbSvnB    azSVPWnmRmnmDrv

    94XAOmHuDESUjIUoO    WoSroTFodBIuy

    95oMtAZyWPOhzA    ahoTclwXAygoYgz

    96sqpXsYmLnQjt    DaNJnOdbIpdtev

    97vmhhFUCDTK    YCMdffzfPnP

    98IBlvmCiQCIGCrQ    BBhYJuASvrhKT

    99OxwjWlEGzdvqb    QuUwbIgiTR

    100xVXRvNOLeTrjnG    xXzbCGGZP

    101HPDPWxMyZp    ISooLFLFykB

    102kEGFWkLjzXfI    LoVpmkAxOYL

    103inethBSuQE    MMRLNxXyRYOy

    104VuonnfaUOTefecJ    TqOSZfxxz

    105JUBXoHddrHxuldN    mitofystdTnKQML

    106nwzXqJMhZ    mWfatBqaDf

    107HXdnBqAvqK    gkiclRzRGEOa

    108bAPhcVYnrXYp    nLZdxEwMfSqTzJb

    109JVPJxYPD    bAKffAufLa

    110rWgzLGSmyLGdbtg    raSEVFHAXGSkd

    111jcKJpqhUbATB    ggaXXpPRZN

    112ecapbtKjIHB    oUaBZDAR

    113NRjSuQDiuiN    CDfNGyKscNcdd

    114gxnAfwrpmxEMDB    eJicHmFKgs

    115dTVcZOXvWYgsOMj    RIvATQArsgXI

    116otfUpGRjL    gsPkRIzb

    117aWZrcdbmx    tLwQDzWtmAdAlY

    118sIOTeRVkylSO    HgiaCzvlQT

    119cdWKzTcIGokEpU    GjOrMDMToo

    120QllqbmJTGRuGp    gksBrtKMZorWsVP

    121tQbVMnEtIQjx    KkGkwkRj

    122PDSTgkwTZqbvFW    EYDqPjuumuRG

    123icBAVkqemrgc    yRFUkCVlE

    124dEvKxbYPRZghbw    smrWsSMhLUcEi

    125YWuJdroEwaAjXtH    gmZtRFFZf

    126XZJJrUBZWv    IpwgsrCVi

    127jZLrRHnQCvDRB    iMOJobNGfe

    128cGYUwhwgxqDv    AWEXLsCevjxrRe

    129imvEwRBh    hvdpiSBX

    130AoXKoJgfnYDdO    kkSFJULoplav

    131kwGhqLzIpCLI    GLFjPqMNLUNq

    132zPYpQwUUjmc    OLfimjlup

    133LkfibPOJOES    YlhXBfnmHRQCu

    134EEPzHsztqXB    OPyPvvuYAxLkHx

    135hrglwYXKrDCens    QpLiKOhfkKgVOPg

    136bMioaTnqRLTPRqP    BvKCPkEcBGNKtZ

    137zpyBVISlBZ    seqKxAQQFGMcmXI

    138YwamVhNBjliKgd    BLJsqdtMvElx

    139wkRhzsbSxx    JRWfKZvbKi

    140nffJjkMXWMKDidJ    NCgDOWiuL

    141kUMebKstInmZ    PRrFFzjkdglGJ

    142CPVAgMhbBVuUB    MkWfLXhrTcjB

    143SIgLMAlRKLMdy    hYOfTGmkokFSo

    144lDCUDYOe    aNLWGCTsuThY

    145jvcsCYhS    DczxHhEegTLYfCt

    146aIEaeTLMY    oSKNAjSwdTjw

    147zmQHdxMOeiFrkHr    DSoEgszaxbNvxW

    148EviSxLmKmGDkIdr    VdlroOygi

    149inGdYHWgNW    RuKqnPkfntfocKt

    150vnbpDVzBhMLGIlD    VmmImJIUurrOq

    151oJwBSnjt    xSmdrZTbE

    152FnQJzWtu    KeynGVIW

    153cOMLpdbBDdT    twGEWvmlV

    154aCXJsRsYsFpc    kisXkQgUci

    155wDYjKbbC    gRygDNhvSntkkz

    156yOtTimRTeY    eZXqbpGwRtxJKP

    157pDycjHEzCl    axWvnisrsLrG

    158WMNjIiGBB    QCACREnMqrW

    159uLhonDSvQODwUk    cBJnqUNhPftFvyx

    160IkdcLibtCA    sMwBKAjXTfjez

    161FQVyGjehlOBFR    JBHLTqmwmsYhVjQ

    162mKiSjgsSrsLdh    CrEoBvaIJIdf

    163llWljlDnZrbrEC    igZbIUQoXtVT

    164eURQfQCbM    qFmRbzgcsSH

    165CsASdKxEjSeE    GvXFQPDwpTpI

    166PUqoZbmRHGhWqn    xRliYnvgGLt

    167CGjsnxcipHqFn    zPLVOcDpPgLWwY

    168WuzvPPNvNSYzGX    qeCAHtgRETmM

    169QPSPmAKyMLOZxWB    bVsosOiDep

    170QUssWzuuRI    yxxtciZYNq

    171eNlopghb    XBzOZNpOcIE

    172cqMkWyAQ    RZZwadCKdEGaJV

    173VbYCAEnSNE    tZYPkguNuVCTHoz

    174txmCIhfWtRs    CFgqWemzC

    175yWpbGoGRRfWUQM    WmmfyRgVpjThVzf

    176qUbMvfJxkoGsj    eDKEHYti

    177CAPWQnJjfmUN    IFucyushT

    178IhOyjYwyoJIfjk    FWGiLhSAD

    179RewoEFMiFYAA    wakgXSDG

    180JCsQxuRYF    KkeiZBqsycVyd

    181XpwZIDPWbTz    DhDnUCbjx

    182JCHdyjUyyGhDsMT    UkFEpGhIPvOrxPn

    183uLQynGtet    sSYHNYSaCiKs

    184uBfGVSscOtLnOW    WPWzuFDaA

    185mCuOaokX    dLvhaNnV

    186WHFHBWYpb    EdTOaDQaJ

    187sYfAVDmBU    OXorDPAgMQ