Re: Question about where to deploy the business logics for data processing

You're gonna lock yourself into SOMETHING, that's why there are still
thousands of COBOL programs still being maintained.

Mike Nolan

On Fri, Jun 9, 2023 at 3:39 PM Ron <ronljohnsonjr@gmail.com> wrote:
>
> You can be sure that banks and academic research projects have different needs.  Heck, your University's class
schedulingsoftware has different needs from the research problems that you support. 
>
> The bottom line is that putting all of the "business" logic in TypeORM locks you into using an ORM, while putting as
much"business" logic in database as stored procedures, triggers, foreign keys, etc... doesn't.  Parts of the
applicationcan be in Java, some in JS, C, C++, Rust, Perl, even COBOL. 
>
> On the other hand, putting so much logic into the database essentially locks you into that RDBMS.
>
>
> On 6/9/23 13:36, Nim Li wrote:
>
> Hello,
>
> Thank you so so much for all the feedback so far.  :D
>
> About this comment:
>
> > "... an application that requires changing the data model does not seem to be well designed...don't allow model
changeby the business logic..." 
>
> I work in a science research faculity.  When researchers start a project, they don't necessary get the full picture
ofwhat they are hoping to achive (yet they may get some ideas about the starting point that allow them to move forward)
By the time they see 40% percent of what they have done, they may start to have a different thought and move towards a
differentdirection, or in some cases, they may spin it off to something different after a certain period of time
Comingwith my Agile Development mindset in the research area, it is common for me to see users changing their
requirementand expectation, with the same buckets for the data.  Yes, there is quite a lot of work to keep the
researchershappy.  ;-) 
>
> I suppose when there is a specific end-goal to achive for a project, a more specific design can be more feasible
basedon the goal.  But when the end-goal is not necessary clear, and/or change-able, I am not exactly clear how we may
drawa black-and-white line to determine a design is good or not (.. and for how long...) 
>
> I imagine one option may be to put less logics and restrictions on the data side, which allows the researchers to
havemore flexible on their end.  But this may not be always feasbile due to the specific protocol of the study.
Perhapsthere may be some other approaches and/or principles to deal with situation like mine? 
>
> My major focus is still on getting more opinions about where to implement the business logics for data processing ...
if you have any thoughts about the design, I would love to hear your thoughts as well. 
>
> Thank you so so much for sharing!
>
> On Fri, Jun 9, 2023 at 12:35 PM Lorusso Domenico <domenico.l76@gmail.com> wrote:
>>
>> Uhm me need to start form 2 concepts:
>>
>> competence
>> Network lag
>>
>> Competence: usually programmers aren't skilled enough about the architectures and the actual needs of each layer.
>> This is a problem, because often programmers try to do something with what he already know (e.g. perform join in
Java....).
>>
>> A correct design requires to identify at least the data logic, the process logic, the business logic and the
presentationlogic. 
>>
>> One of the most important goals of Data logic is to ensure the correctness of data from many point of view (all is
impossible).
>>
>> That involve:
>>
>> audit information
>> bitemporal management
>> strictly definition and verification of data (foreign key, checks, management of compatibility)
>> replicate consistently data for different usage
>> isolate access for actual needs
>> design
>>
>> So an application that requires changing the data model does not seem to be well designed...
>>
>> Network lag
>> The first problem is latency, I must minimize the passage of data over the network.
>> This means, for example, creating a service that allows the caller to choose only the information it needs.
>> But it also means, to get all the information needed in a single call, design asynchronous service, use cache data
physicallynear to the frontend or the middle layer. 
>>
>> Based on these 2 concepts I suggest:
>>
>> develop the Data logic near or inside the database;
>> design powerful and addictive api;
>> don't allow model change by the business logic
>> organize/copy data in jsonb with a powerful json schema to provide coherence through every layer
>> ensure a system to grant ACID features to your process.
>>
>>
>>
>> Il giorno ven 9 giu 2023 alle ore 05:22 Nim Li <mr.nim.li@gmail.com> ha scritto:
>>>
>>> Hello.
>>>
>>> We have a PostgreSQL database with many tables, as well as foreign table, dblink, triggers, functions, indexes,
etc,for managing the business logics of the data within the database.  We also have a custom table for the purpose of
trackingthe slowly changing dimensions (type 2). 
>>>
>>> Currently we are looking into using TypeORM (from Nest JS framework) to connect to the database for creating a BE
thatprovides web service.  Some reasons of using TypeORM are that it can update the database schema without any SQL
codes,works very well with Git, etc.  And from what I am reading, Git seems to work better with TypeORM, rather than
handlingindividual batch files with SQL codes (I still need to find out more about this)  Yet I do not think the ORM
conceptdeals with database specify functions, such as dblink and/or trigger-function, etc, which handles the business
logicsor any ETL automation within the database itself (I should read more about this as well.) 
>>>
>>> Anyway, in our team discussion, I was told that in modern programming concept, the world is moving away from
deployingprogramming logics within the database (eg, by using PL/SQL).  Instead, the proper way should be to deploy all
theprogramming logics to the framework which is used to connect to the database, such as NestJS in our case.  So, all
weneed in a database should be only the schema (managed by ORM), and we should move all the existing business logics
(currentlymanaged by things like the database triggers, functions, dblink, etc.) to the Typescript codes within the
NestJSframework. 
>>>
>>> I wonder if anyone in the community has gone through changes like this?  I mean ... moving the business logics from
PL/SQLwithin the database to the codes in NestJS framework, and reply on only the TypeORM to manage the update of the
databasewithout any SQL codes?  Any thoughts about such a change? 
>>>
>>> Thank you!!
>>>
>>
>>
>> --
>> Domenico L.
>>
>> per stupire mezz'ora basta un libro di storia,
>> io cercai di imparare la Treccani a memoria... [F.d.A.]
>
>
> --
> Born in Arizona, moved to Babylonia.