PDA

View Full Version : How is persistence achieved in the language you use!


Zen
February 9th, 2004, 02:20 PM
Many 4GLs provides two ways to run a program-
1) Normal way to run it and 2) Program runs persistently.

When a program is run normally (non-persistent), it does not stay in the memory of the computer after it’s execution is over, while a persistent program stays in the memory after its execution is over and it can be invoked again by referring to its handle. Even an internal procedure can be run by making an appropriate call. The Publish-subscribe mechanism relies on such persistent procedures.

My question is – How is persistence achieved in VB, JAVA, C++ or .NET – whichever language you are familiar with! One or more small examples shell be good.

I have a second part of this question, after this one is answered.

GunsNRoses
February 9th, 2004, 03:02 PM
What you're describing sounds like a Terminate and Stay Resident (i.e.) a TSR program.

Persistence is supported in Python. Mostly, it consists of serializing an object and dumping that to disk. Later on, you can read that file back into memory and voila your object is back in the exact state that you left it in. It's easier with bytecoded languages I guess, though other languages do support persistence, if you have serialization. As for some sample code, here's some using Python's cPickle module (python has several modules to do serialization)

========= Save dictionary using cPickle =======
#!/usr/bin/env python

import cPickle

data = {1: 'foo', 5: 'bar', 42: ('foo', 'bar')}

#Write out data
out = open('output.dat', 'wb')
cPickle.dump(data, out);
out.close()

# read it back in
inp = open('output.dat', 'rb')
data2 = cPickle.load(inp)
inp.close()

# print the data
print data2


Also see this article: http://www-106.ibm.com/developerworks/linux/library/l-pypers.html

ShivSainik
February 9th, 2004, 08:17 PM
Originally posted by Zen


My question is – How is persistence achieved in VB, JAVA, C++ or .NET – whichever language you are familiar with! One or more small examples shell be good.



I am not sure whether you can have persistent programs in Java where they leave their footprint in memory after execution is over and then you can invoke them back. (Atleast I know about C++ and Java)

As said by GNR, you can have TSR where program is persistently in the memory, but again I don't think it matches exactly with your description. (i.e. calling specific method anytime using some mechanism)

GunsNRoses
February 9th, 2004, 08:29 PM
Heh, you can call methods on a TSR. I used to do it by using an unused (or rarely used) interrupt vector to point to my TSR program.

To talk to my TSR from a regular program, I could put values into the CPU registers and then issue that interrupt. My interrupt handler on the TSR would read the value of the AH register and decide what function to execute. Depending on the function, it would also read the values on the other registers. Then it would execute the necessary function and then return.

Zen
February 9th, 2004, 10:21 PM
Yes GunsNroses,

The concept is exactly the same! it is just a TSR but in 4gl the whole thing is managed for you ... I do remember how we used to trap a key combination to kick the TSR written in c back in action. :)

Thanks for the article link - it is so well written.

I hope to get more responses.

Now what made me ask this question?

A VB program or a set .NET assembly file is a .DLL.

I was wondering how persistence is achieved in VB also and does packaging the program module has a realtion with it.

is handling the .DLL some kind of automated TSR management on the behalf of the application?

Is any other word is used for the similar functionality?

I wish some ustad familiar with both the concepts and internals could clear the fog for me :)

ShivSainik
February 9th, 2004, 10:24 PM
Originally posted by GunsNRoses
Heh, you can call methods on a TSR. I used to do it by using an unused (or rarely used) interrupt vector to point to my TSR program.

To talk to my TSR from a regular program, I could put values into the CPU registers and then issue that interrupt. My interrupt handler on the TSR would read the value of the AH register and decide what function to execute. Depending on the function, it would also read the values on the other registers. Then it would execute the necessary function and then return.

Right, thats cool....

Thorn
February 10th, 2004, 07:43 AM
Zen, TSR was a teminology used in the olden days when the application was supposed to be running in the memory even after it was terminated (or IRQ was handled back to the command shell) in old DOS days when running more than one program was not possible to handle by the OS.

In the present world, we have operating systems capable of running multiple aplications or processes simultaneously and also multiple instances of the same process or applications. Behind the scenes these are handled as "Threads". You can program in .NET (I'm sure you can do that in other 4GLs also), where you create a main thread and call another thread from the main and then kill the main thread, letting the called thread to live. This called thread can call other methods or functions... giving you the feel of Terminate and Stay Resident. Another approach is creating Windows Services which run in the background and starts automatically when the PC is started and does not need authentication to login... doing its job and still keeping the OS capable of doing other things

Serializing an object and storing it and calling back when you restart the application cannot be put into the category of TSRs as you're storing the information in a file and when the application terminates, OS looses the control to that process or application. I guess we're talking of keeping the process alive even when the application calling that process dies.

In a nutshell I can think of 2 solutions to handle TSR in the present world: Threads and Windows Services.

Big-G
February 10th, 2004, 08:55 AM
Originally posted by Thorn
Serializing an object and storing it and calling back when you restart the application cannot be put into the category of TSRs as you're storing the information in a file and when the application terminates, OS looses the control to that process or application. I guess we're talking of keeping the process alive even when the application calling that process dies.


I agree. Serialization just stores the state of an object to some form of persistent storage (file/object database etc). This is not the same as a persistent process. But if this is what Jhandoo was asking, then it's very very simple to do in Java. All you have to do is to have your class implement an interface called Serializable. This is the simplest of interfaces which just has a flag and no methods that need to be implemented. You write the object to the disk using ObjectOutputStream class and read it back using ObjectInputStream.

But Jhandoo....i think you are referring to Daemon processes. Processes that run perpetually and can 'listen' to events/signals/interruptions.

Regardless of the language, the only way to do this is to have an infinite loop in your program at some level [while(true)], which sleeps between every iteration and also listens to events/signals before resuming the next iteration. That's how all the daemon processes are written. And this is possible in almost all programming languages. All java containers (JSP, servlets, EJB), webservers, mail servers use the same funda.

In java for invoking methods (or procedures as your call them) on an object persisting in a seperate process (which means a seperate JVM), the technology used is called RMI. Once implemented, it's as simple as calling methods from a local object.

GpeL
February 10th, 2004, 08:58 AM
While true loop and within the program event handlers or polling (periodic listening to some port) as Bigs suggests.

My implementation is purely event driven. (Written in good ol.. C).

Zen
February 10th, 2004, 09:13 AM
If All I have is a hammer everything looks like a nail. So I wanted to see other method of achieving persistence in different languages. This thread became more valuable than I thought.

In the 4gl I use, persistence is kind of simple to implement.

say I have 3 programs -

caller-prg
called-prg-1
called-prg-2

there is a RUN statement.

code in caller-prg would look something like -



define variable lhProcHdl as handle.

/* non persistent call */
run called-prg-1.

/* persistent call */
run called-prg-2 persistent set lhProcHdl.



That's it ... the called-prg-2 stays in memory after it returns controls to the caller-prg. now any other program called from caller can use the handle and call any internal procedure within called-prg-2 with some code like -



....
...
/* run an internal procedure named ipMyFunc in called-prg-2 */
run ipMyFunc in lhProcHdl.
...
...


Ofcourse 4gl makes my life simpler. I think internally threading is implemented by the language - can't think of any other way.

This persistent procedure is available for the session of the application or the caller program depending on your syntax (not going in those details right now) But not for the session of the windows. A service on ther other hand is available for the session of the windows if not forced to terminate. :)

I do remember thought, the .DLLs were initially seen as an automated alternative to TSRs by many application developers.

Thorn
February 10th, 2004, 11:44 AM
Originally posted by Zen

I do remember thought, the .DLLs were initially seen as an automated alternative to TSRs by many application developers.

See... the DLLs are never persistant objects. They are set of library routines or business logic stored in a file or bunch of files. When your procedure calls these DLLs then they do the work what they're supposed to do. If your PC has a bunch of DLLs, they're not doing any job or running any processes all by themselves. They come into action only when called either by a persistant object or non-persistant object.

Zen
February 10th, 2004, 12:15 PM
Originally posted by Thorn
See... the DLLs are never persistant objects. They are set of library routines or business logic stored in a file or bunch of files. When your procedure calls these DLLs then they do the work what they're supposed to do. If your PC has a bunch of DLLs, they're not doing any job or running any processes all by themselves. They come into action only when called either by a persistant object or non-persistant object.

Exactly, 'alternative' I had said! :)

quote

CLIENT CONFIGURATION UTILITY

The SQL Client Configuration Utility is installed as one of the SQL Tools programs. It is used by Windows NT server clients, Windows NT workstation clients, Windows 95 clients, and Windows 3.x clients to view and modify the following components:


DBLIB versions (for example, Ntwdblib.dll)
Net library .dlls (for example, Dbnmpntw.dll)
Server Connection Information (if necessary)
DOS clients do not use this program to establish connection parameters. Instead, a terminate-and-stay-resident (TSR) program is run once to provide the same information. It is best to add a line to the Autoexec.bat file to run this TSR or add it to a .bat file before starting your application. The TSR to run is located in the MSSQL\Bin folder and depends upon your network protocol, as listed in Table 3.3.

http://www.windowsitlibrary.com/Content/77/03/2.html


I still wonder how persistence is achieved in VB and .NET ... are services or multi-threading the most popular methods?

Thorn
February 10th, 2004, 01:31 PM
Usually the services created in .Net uses 2 threads. One is the main thread that listens to the events like stopping, disabling and starting the service and the other is the worker thread that is responsible for doing what the service is targetted at doing. This thread (worker thread) is executed for a pre-determined time period and on each loop, it checks with the main thread to see if it has to respond to any of the events triggered by the main thread. If the main process of the worker thread takes long enough to react to the main thread's events, it keeps checking the status asynchronously. This all depends on the developer, how he wants to handle most of this.

Yes! the kind of "persistance", you're referring to is handled by multi-threading in VB and .Net

Zen
February 10th, 2004, 01:59 PM
Originally posted by Thorn
Usually the services created in .Net uses 2 threads. One is the main thread that listens to the events like stopping, disabling and starting the service and the other is the worker thread that is responsible for doing what the service is targetted at doing. This thread (worker thread) is executed for a pre-determined time period and on each loop, it checks with the main thread to see if it has to respond to any of the events triggered by the main thread. If the main process of the worker thread takes long enough to react to the main thread's events, it keeps checking the status asynchronously. This all depends on the developer, how he wants to handle most of this.

Yes! the kind of "persistance", you're referring to is handled by multi-threading in VB and .Net

"Thenks"!

GpeL
February 11th, 2004, 09:21 AM
Zen,

My server runs on SOLARIS so I dunno how its done on widozz platform. Here is my server design in a nut shell.

Each client connection to my server creates a new thread. Each of these threads are assigned a message queue in the servers' shared memory pool. Each thread has its message queue identifier and that way it can only check/change the contents of its own message queue (ofcourse thru server calls.. like get_thread_info, send_thread_info and such calls).

So lets say the client initiates a command (My server has extended sql commands that allow it to give you ODBC compliant data from any source including flat files and it can output in many formats csv, tab, space, fixed file, and ODBC compliant so you can query it like any standard database but that is not the purpose of the server cos it is an app server not a dataserver).

This server runs 24 X 7 so its persistant as you call it.
First a client connects to the server. That is a connect event. The client gets a thread a message queue id.

Now for example, a client sends out a sql. "select name from sysobjects" , this is an event. A language event. The event and the command is placed in the event queue. The server then goes and processes the command and puts the result set in the queue puts the format specifier, the column details, the datatype detail in the message queue and executes a send event. This event tells the client to pick up the results.

Finally when the client disconnects, it initiates a disconnect event when the thread is destroyed and the message queue is removed from the memory pool.

There are many events but the basic building block is same.. be it an RPC event, or any other extended command set event.

There are various other components.. like deadlock handler, thread scheduler et. al. but you get the general idea.