Sumo Logic, Jason Data, Sim Security discussed on Software Engineering Daily

Automatic TRANSCRIPT

I would closely related to Sim as much as I don't like the connotation that acronym generally has but I think in Sim. Sim Security in phone. Event Management Okay. Yeah but I think you could also compare like oh seeing the beginning. Most people use spunk in elastic. Search for this purpose so generally were compared against that but depending on where you folks in the product. So if you're looking at compliance features we get compared against cloud security vendors like red lock or even evident evident which you know was acquired by Palo Lotto it really depends on what the team is emphasizing. What they'll use the product for but for the primary purposes for doing log analysis in which case compared into traditional sims? Like God forbid you know if people are still using like these really really older Sims. But I'd say primarily people just build a semi top of spunk. That's usually like nine. Ten people are doing or I guess now. Sumo logic has a slightly easier platform than spunk does. But in my mind mind. They're so similar and does that differ from your product. The Panther log monitoring product because in the sumo logic or spunk world. They're hosting your logs. Also right like you're shoving your logs into their systems so the panther world is more decoupled right. Because you're not actually restoring them you're just running analysis over there logs and they may be able to just keep their logs in on their own cloud cloud servers right doesn't necessarily need to be exported or do understand the product. Yeah all explain it really quickly so I think with sumo logic they only we have a saas implementation. Sea shipping the logs. It lives in one of their server somewhere. And then they give you a log in where you can kind of go through. The data and spunk has a similar cloud offering. They think primarily due on prem. Stalled appointments. Were the data stays within the premise of the company. And that's the Z.. Model that Panther has so with Panther you would deploy within your one of your. Aws accounts generally our recommendation. Is You create a sub account in the deploy panther within within that. And then. That's the account where you tend all your security data to. That's the account that you would run the panther application itself. which will normalize the data make? Sure it's in the correct state state. Ill ill store in three and then it will. Do you know we can use that. Data for compliance we can use it for logging. We can also search over historically and that's really the area that we're going invest more in as well as like. How can we understand the data better so take me through the full data pipeline from like some event that hits and Application Service to the log message from that service being generated to the log message being stored or just shuttled through panther? I just want to understand the full data pipe litter. Yeah so from the source Let's just say that. We're looking at security focused applications. Meaning there's tools like Os query which is operating system level instrumentation and and always career. We'll tell you which users have installed on your system which programs you have installed like. What are some running processes? It'll just gives gives you any sort of generalized information and with these tools. Data can really be in any format so it could be Jason Data could be data could be some random custom format you know with the case of SIS log for example that could be literally whatever the administrator configures it to be so there can be a lot of challenges Andrea there but the first step in the pipeline is really just normalization taking the data and putting it all on Jason and the reason we have to do that is because of the rules they taken a event argument and then we write rules on it expects it to be in a certain format so we need to be normalized. I as JAS on then we can rules was on it. And then we can generate alerts the way the pipeline works is the data's generated it's aggregated it's put into something that's in the aws ecosystem whether it's a Q.. Hugh or an S. three bucket or can stream and you know the recommendation. We usually have is like probably easiest way to get all this done and most reliable. We'll just use US three. We pick it up from three. We normalize it and then we send through a series of pipelines like lambda functions. That eventually have it land into a different S. three bucket in the right format and then from there we do all our analysis on the data so we bash data we analyzed allies. Like let's say like a thousand lines at a time right and then any alerts are generated sends into different pipeline. And then maybe there's another fork of the pipeline that says okay. Let's let's use compliance. Let's scan and EC two instance and make sure that all of these little attributes. Are you know the right setting. And that's a common complaints workflows well and then you know if there's an issue there trigger another pipeline so like Byu Lambda that you can tell these really complex architectures but the data comes in normalized analyzed and then alerts send off and then someone gets a text or phone call on their phone they click a link it opens them into the Panther. Ui They can say okay. This alert was generated. This is the full log file and now we can go search our logs and find even more information. Listen and we're searching logs. The backend for that is you know is S. three and Athena and you know we're using these normalized allies logs that we set up to empower that pipeline. Can you give me an example of how customers use it. Like maybe a customer use case. That exemplifies why panthers US wall yeah so I would say on boarding any aws level log data and then immediately getting alerts from it. So let's say I was mentioning before we have a bunch of built-in rules to platform uh-huh and let's say they're already collecting their clinical data which is like a really common use case and everyone should be doing this so they do is they they would you know they were installed the panther application some account. They Will Linka S. three bucket to it and then we would start analyzing this three logs and then we could immediately tell you like oh this activities happening right. Now it's really from this point onward and then alternatively we can also use the same clock data to start to scan your infrastructure to say. Okay you actually have these five S. three buckets that you should look at right now as well. These are public buckets or they're not following the encrypted or or you know other configuration would wanNA check and that's kind of how the customer journey starts. It's like we feed in one day to source out of time. We see what alerts generated we tune the alerts and then we add more. And that's the pattern. We follow AIRBNB as well. We would start with some data we tune we would ride more rules who cover all these different use cases aces and then we'd move onto onboard in more and more and more data and the more data. We collect the more well rounded view that we have of everything the way I always explain. It is like you do cloud based environment data which is like your cloud trail or you're like infrastructure level logs and then you move down the stack to like network network logs. So you know layer three Netflix data or if you have layer seven full information on requests going certain websites and their arguments and cookies and all that more rich information and the moving down the stock further you have host level information so this is stuff like was this queries can be like databases the things that are running the applications which is the next level and then applications could be like your web application it could be like tools like Os was query or Santa which is a binary wireless tool that was developed by google. And you take all this context and then you put it into a single place. Listen then that's how you get the unified view of everything take. Apache Cassandra.

Coming up next