Quick Tests

Test Seagence using Sample Ecommerce Application.

Follow below steps to see Seagence in action on Broadleaf Commerce’s sample Heat Clinic application.

Pre-requisites: JDK 17, Maven 3.9.6, git command line tool.

1) Create Seagence Account

Click “START FREE TRIAL” button at the top right corner

Register on app.seagence.com.
Login with new root user name/password.
After successful login, go to Settings tab, click “Download SGAgent” to download sg-java-agent.zip.
Unzip the contents of sg-java-agent.zip to a directory of your choice. You will find 2 files in the zip, sg-java-agent.jar and sgagent.properties.
In the same Settings tab, click “API Token” and copy displayed API Token.

2) Connect Datadog Account (Optional)

1) Login to your Datadog account.
2) Go to Integrations page and search for Seagence.
3) Click “+ Install“, then click “Install Integration” in the popup window. This will install Seagence integration.
4) On the Seagence Integration popup window, in the Configure tab, click “Connect Accounts“.
5) You will be redirected to SeagenceWeb login page. Login to SeagenceWeb using your Seagence account.
6) You will now be redirected to Datadog authorization page. Click “Authorize” to give access to the required scopes.
7) You will now be taken to Integrations page on SeagenceWeb showing successful Datadog integration.

This integration will post Seagence detected defects to Datadog.

3) Clone Git Repo with Test Defects

1) “git clone https://github.com/seagence/DemoSite.git” into a new directory

2) Update seagence.java.agent property value in the root pom.xml file with the location of sg-java-agent.jar and sgagent.properties. This will attach sg-java-agent.jar to Heat Clinic application.

3) Now update sgagent.properties file with your userName and apiToken. Also assign an applicationName, and hostName. No need to change seagenceWebURL.

Note:

It is important to mention that Seagence detects defects that are caused by an exception. Seagence will not be able to detect defects that are not caused by exception. For example, Seagence will not be able to detect defect that displays the price of an item as $20 instead of $10, because such a defect, most likely, occurs because of db mis-configuration. Seagence detects defects caused by any exception (can be caught, uncaught, or swallowed exception or any other exception) because an exception disrupts request execution which reflects in the ExecutionPath Seagence records.

Also, Seagence makes one assumption. To detect defect, Seagence assumes that majority transactions of the end point are successful. This assumption is safe as production applications have 99% success transaction rate. As a starting point, for these tests, make sure 15% of the requests encounter defect and 85% become success.

A) Set Up Multithreading Defect

Troubleshooting intermitently occurring multithreading defects is difficult. If the exceptions that result due to the multithreading defect are swallowed and return 200 http success code, leaving no trace in the logging system, the troubleshooting becomes even more difficult. The exceptions raised due to this multithreading defect are swallowed and return 200 http success code but Seagence still detects this defect.

1) Copy the contents of DemoSite/site/src/main/resources/CartController_With_Multithreading_Defect.java.java into DemoSite/site/src/main/java/com.community.controller.cart.CartController.java to add the multithreading defect in the “/add” end point. This defect is introduced using apache’s XML DocumentBuilder, which is not thread safe. Note that /add end point is fetching messages from readMessage() using non-thread safe DocumentBuilder. When parallel threads race and access this shared DocumentBuilder object, the object state corrupts and results in request failures.

2) Build: cd to DemoSite and issue “mvn clean install“.

3) Run: Now start site application. cd to site and issue “mvn spring-boot:run“. Note: Broadleaf commerce uses Solr internally. If you run into any Solr error, do “mvn spring-boot:run” again to resolve the error.

Testing:

This test will create number of virtual users to send parallel and consecutive requests to “/add” end point of the Heat Clinic application on port 8083. If the application is configured to run on a different port, change the port in com.sg.util.VirtualUsersForMultithreadingDefect.java file.

1) git clone https://github.com/seagence/SeagenceMultithreadingClient.git

2) To build

cd SeagenceMultithreadingClient

mvn clean package

3) To Run

java -jar .\target\seagence-multithreading-defect-client.jar <8> <500>

This client takes two integer program arguments to the main method. The first argument tells how many virtual users or threads to create. The second argument tells total number of requests to send. Both arguments are optional, in such case 8 and 500 become default. On a Windows laptop 6 to 8 parallel threads is good to produce this multithreading defect but this also depends a lot on the machine being used for the test. So feel free to adjust these values.

After execution completes, goto Baseline tab on SeagenceWeb and click “Create Baseline” button to create baseline from the data collected. Note that, in a production environment, baseline creation happens automatically at regular intervals. For the purpose of this test, instead of waiting for Seagence to create the baseline, you create the baseline now. Baselining will take few momenents. After that you will receive email alert for the multithreading defect. Also goto Clusters tab on SeagenceWeb and notice the red cluster with defects. From now on, if users encounter this multithreading defect, Seagence will compare corresponding trace against the baseline and will come to know it’s a defect and alert in realtime. Go ahead and run the test program again, this time with reduced number of requests, “java -jar .\target\seagence-multithreading-defect-client.jar 8 100”. During this second test, for example, if 10 requests fail with multithreading defect, you will receive 10 email alerts. Note, to stop receiving too many emails, you can set the the “Real Time Alerting” flag to false in Settings -> Settings.

B) Set Up Hidden Defect

If you have already done the above test, start afresh. Goto Settings and “Delete Data”.

This defect is caused by a swallowed exception and returns a 200 http success code to the browser. This means that this defect leaves no trace in the logging system and makes troubleshooting difficult.

1) Copy the contents of DemoSite/site/src/main/resources/CartController_With_Hidden_Defect.java.java to DemoSite/site/src/main/java/com.community.controller.cart.CartController.java to add the defect. The defect is introduced in “/add” end point for the sause product “Armageddon The Hot Sauce To End All” (productId=9) by setting productId to null.

2) Build: cd to DemoSite and issue “mvn clean install“.

Testing: The minimum number of traces needed per end point to create Seagence Clusters and detect defects is 100. So, goto http://localhost:8083/ and add all Sauce products on the Home page and on the Hot Sauces page to the cart. While you do that, notice that, when you add “Armageddon The Hot Sauce To End All” sause, the item will not add to the cart but browser receives a 200 success http code. Now, remove all items from the cart and add all sauce items again. This need to be repeatedly done until the end point receives 100 transactions. This exercise takes about 5 minutes. You can find the count of traces collected in the Baseline tab -> View Data. If the count not reached 100, add few more itmes.

Now click “Create Baseline” button to create baseline from the data collected. Once baseline is created you will receive email alert for the hidden defect. Also go to Clusters tab on SeagenceWeb and notice the red cluster with defects. From now on, if users encounter this hidden defect, Seagence will compare the corresponding trace against the baseline and will come to know it’s a defect and alert in realtime. Go ahead and add “Armageddon The Hot Sauce To End All” item to cart. You will encounter defect and Seagence will detect it in realtime.

Note: The “Delete Data” function in Settings tab on SeagenceWeb will wipe out all your data. If you are playing around and want to start with a clean slate, feel free to Delete Data. If you do that, you will have to restart your application.

Now play around with the application by introducing new defects.

Conclusion

Notice that, during this testing

1) No configurations like alert rules, threasholds, suspicious exceptions, and/or code changes etc were set. Which means that Seagence detected these defects autonomously.
2) After initial baselining, Seagence detected defects within 5 seconds of the end user encountering the defect. Which means that Seagence detected defects automatically in real time.
3) You were provided with the correct root cause of the defect, even if the system swallowed the root cause exception, leaving no trace in the logging system, and returning a 200 success code.
4) You got to the root cause without doing any troubleshooting.