using DotNetCore.CAP; using JiShe.CollectBus.Ammeters; using JiShe.CollectBus.Common.BuildSendDatas; using JiShe.CollectBus.Common.Consts; using JiShe.CollectBus.Common.Enums; using JiShe.CollectBus.Common.Extensions; using JiShe.CollectBus.Common.Helpers; using JiShe.CollectBus.GatherItem; using JiShe.CollectBus.IoTDBProvider; using JiShe.CollectBus.IotSystems.MessageIssueds; using JiShe.CollectBus.IotSystems.MeterReadingRecords; using JiShe.CollectBus.IotSystems.Watermeter; using JiShe.CollectBus.Protocol.Contracts; using JiShe.CollectBus.Repository.MeterReadingRecord; using Microsoft.Extensions.Logging; using System; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Threading.Tasks; namespace JiShe.CollectBus.ScheduledMeterReading { /// /// 定时采集服务 /// public abstract class BasicScheduledMeterReadingService : CollectBusAppService, IScheduledMeterReadingService { private readonly ILogger _logger; private readonly ICapPublisher _producerBus; private readonly IIoTDBProvider _dbProvider; private readonly IMeterReadingRecordRepository _meterReadingRecordRepository; public BasicScheduledMeterReadingService( ILogger logger, ICapPublisher producerBus, IMeterReadingRecordRepository meterReadingRecordRepository, IIoTDBProvider dbProvider) { _producerBus = producerBus; _logger = logger; _dbProvider = dbProvider; _meterReadingRecordRepository = meterReadingRecordRepository; } /// /// 系统类型 /// public abstract string SystemType { get; } /// /// 应用服务器部署标记 /// public abstract string ServerTagName { get; } /// ///电表日冻结采集项 /// protected List DayFreezeCodes = new List() { "0D_3", "0D_4", "0D_161", "0D_162", "0D_163", "0D_164", "0D_165", "0D_166", "0D_167", "0D_168", "0C_149", }; /// /// 电表月冻结采集项 /// protected List MonthFreezeCodes = new List() { "0D_177", "0D_178", "0D_179", "0D_180", "0D_181", "0D_182", "0D_183", "0D_184", "0D_193", "0D_195", }; /// /// 获取采集项列表 /// /// public virtual Task> GetGatherItemByDataTypes() { throw new NotImplementedException($"{nameof(GetGatherItemByDataTypes)}请根据不同系统类型进行实现"); } /// /// 构建待处理的下发指令任务处理 /// /// public virtual async Task CreateToBeIssueTasks() { var redisCacheKey = $"{RedisConst.CacheBasicDirectoryKey}{SystemType}:{ServerTagName}:TaskInfo:*"; var taskInfos = await FreeRedisProvider.Instance.KeysAsync(redisCacheKey); if (taskInfos == null || taskInfos.Length <= 0) { _logger.LogWarning($"{nameof(CreateToBeIssueTasks)} 构建待处理的下发指令任务处理时没有缓存数据,-101"); return; } foreach (var item in taskInfos) { var tasksToBeIssueModel = await FreeRedisProvider.Instance.GetAsync(item); if (tasksToBeIssueModel == null) { _logger.LogWarning($"{nameof(CreateToBeIssueTasks)} 构建待处理的下发指令任务处理时Key=>{item}没有缓存数据,102"); continue; } //item 为 CacheTasksToBeIssuedKey 对应的缓存待下发的指令生产任务数据Redis Key tempArryay[0]=>CollectBus,tempArryay[1]=>SystemTypeConst,tempArryay[2]=>TaskInfo,tempArryay[3]=>表计类别,tempArryay[4]=>采集频率 var tempArryay = item.Split(":"); string meteryType = tempArryay[3];//表计类别 int timeDensity = Convert.ToInt32(tempArryay[4]);//采集频率 //获取缓存中的电表信息 var redisKeyList = $"{string.Format(RedisConst.CacheMeterInfoKey, SystemType, meteryType, timeDensity)}*"; var oneMinutekeyList = await FreeRedisProvider.Instance.KeysAsync(redisKeyList); if (oneMinutekeyList == null || oneMinutekeyList.Length <= 0) { _logger.LogError($"{nameof(CreateToBeIssueTasks)} {timeDensity}分钟采集待下发任务创建失败,没有获取到缓存信息,-103"); return; } if (meteryType == MeterTypeEnum.Ammeter.ToString()) { // 解析结果(结果为嵌套数组) var meterInfos = await GetMeterRedisCacheData(oneMinutekeyList, $"{timeDensity}", meteryType); if (meterInfos == null || meterInfos.Count <= 0) { _logger.LogError($"{nameof(CreateToBeIssueTasks)} {timeDensity}分钟采集待下发任务创建失败,没有获取到缓存信息,-104"); return; } await AmmerterScheduledMeterReadingIssued(timeDensity, meterInfos); } else if (meteryType == MeterTypeEnum.WaterMeter.ToString()) { //todo 水表任务创建待处理 //await WatermeterScheduledMeterReadingIssued(timeDensity, meterInfos); } else { _logger.LogError($"{nameof(CreateToBeIssueTasks)} {timeDensity}分钟采集待下发任务创建失败,没有获取到缓存信息,-105"); } _logger.LogInformation($"{nameof(CreateToBeIssueTasks)} {timeDensity}分钟采集待下发任务创建完成"); //删除已经处理过的缓存数据 await FreeRedisProvider.Instance.DelAsync(item); } } #region 电表采集处理 /// /// 获取电表信息 /// /// 采集端Code /// public virtual Task> GetAmmeterInfoList(string gatherCode = "") { throw new NotImplementedException($"{nameof(GetAmmeterInfoList)}请根据不同系统类型进行实现"); } /// /// 初始化电表缓存数据 /// /// 采集端Code /// public virtual async Task InitAmmeterCacheData(string gatherCode = "") { var meterInfos = await GetAmmeterInfoList(gatherCode); if (meterInfos == null || meterInfos.Count <= 0) { throw new NullReferenceException($"{nameof(InitAmmeterCacheData)} 初始化电表缓存数据时,电表数据为空"); } //获取采集项类型数据 var gatherItemInfos = await GetGatherItemByDataTypes(); if (gatherItemInfos == null || gatherItemInfos.Count <= 0) { throw new NullReferenceException($"{nameof(InitAmmeterCacheData)} 初始化电表缓存数据时,采集项类型数据为空"); } List focusAddressDataList = new List();//用于处理Kafka主题分区数据的分发和处理。 //根据采集频率分组,获得采集频率分组 var meterInfoGroupByTimeDensity = meterInfos.GroupBy(d => d.TimeDensity); foreach (var itemTimeDensity in meterInfoGroupByTimeDensity) { //将表计信息根据集中器分组,获得集中器号 var meterInfoGroup = itemTimeDensity.GroupBy(x => x.FocusAddress).ToList(); foreach (var item in meterInfoGroup) { if (string.IsNullOrWhiteSpace(item.Key))//集中器号为空,跳过 { continue; } var redisCacheKey = $"{string.Format(RedisConst.CacheMeterInfoKey, SystemType, ServerTagName, MeterTypeEnum.Ammeter, itemTimeDensity.Key)}{item.Key}"; #if DEBUG //每次缓存时,删除缓存,避免缓存数据有不准确的问题 await FreeRedisProvider.Instance.DelAsync(redisCacheKey); #else //每次缓存时,删除缓存,避免缓存数据有不准确的问题 await FreeRedisProvider.Instance.DelAsync(redisCacheKey); #endif Dictionary keyValuePairs = new Dictionary(); foreach (var ammeter in item) { //处理ItemCode if (string.IsNullOrWhiteSpace(ammeter.ItemCodes) && !string.IsNullOrWhiteSpace(ammeter.DataTypes)) { var itemArr = ammeter.DataTypes.Split(',').ToList(); #region 拼接采集项 List itemCodeList = new List(); foreach (var dataType in itemArr) { var excludeItemCode = "10_98,10_94";//TODO 排除透明转发:尖峰平谷时段、跳合闸,特殊电表 var gatherItem = gatherItemInfos.FirstOrDefault(f => f.DataType.Equals(dataType)); if (gatherItem != null) { if (!excludeItemCode.Contains(gatherItem.ItemCode)) { itemCodeList.Add(gatherItem.ItemCode); } } #region 特殊电表采集项编号处理 if (itemArr.Exists(e => e.Equals("95"))) //德力西DTS { itemCodeList.Add("10_95"); } if (itemArr.Exists(e => e.Equals("109")))//WAVE_109 { itemCodeList.Add("10_109"); } #endregion } #endregion ammeter.ItemCodes = itemCodeList.Serialize();//转换成JSON字符串 if (!string.IsNullOrWhiteSpace(ammeter.ItemCodes)) { ammeter.ItemCodes = ammeter.ItemCodes.Replace("WAVE_109", "10_109"); } } keyValuePairs.TryAdd($"{ammeter.ID}", ammeter); } await FreeRedisProvider.Instance.HSetAsync(redisCacheKey, keyValuePairs); } //在缓存表信息数据的时候,新增下一个时间的自动处理任务,1分钟后执行所有的采集频率任务 TasksToBeIssueModel nextTask = new TasksToBeIssueModel() { TimeDensity = itemTimeDensity.Key, NextTask = DateTime.Now.AddMinutes(1) }; var taskRedisCacheKey = string.Format(RedisConst.CacheTasksToBeIssuedKey, SystemType, ServerTagName, MeterTypeEnum.Ammeter, itemTimeDensity.Key); await FreeRedisProvider.Instance.SetAsync(taskRedisCacheKey, nextTask); } _logger.LogInformation($"{nameof(InitAmmeterCacheData)} 初始化电表缓存数据完成"); } /// /// 1分钟采集电表数据,只获取任务数据下发,不构建任务 /// /// public virtual async Task AmmeterScheduledMeterOneMinuteReading() { //获取缓存中的电表信息 int timeDensity = 1; var currentTime = DateTime.Now; var redisKeyList = GetTelemetryPacketCacheKeyPrefix(timeDensity, MeterTypeEnum.Ammeter); var oneMinutekeyList = await FreeRedisProvider.Instance.KeysAsync(redisKeyList); if (oneMinutekeyList == null || oneMinutekeyList.Length <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集电表数据处理时没有获取到缓存信息,-101"); return; } //获取下发任务缓存数据 Dictionary> meterTaskInfos = await GetMeterRedisCacheData(oneMinutekeyList, timeDensity.ToString(), MeterTypeEnum.Ammeter.ToString()); if (meterTaskInfos == null || meterTaskInfos.Count <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集电表数据处理时没有获取到缓存信息,-102"); return; } List meterTaskInfosList = new List(); //将取出的缓存任务数据发送到Kafka消息队列中 foreach (var focusItem in meterTaskInfos) { foreach (var ammerterItem in focusItem.Value) { var tempMsg = new ScheduledMeterReadingIssuedEventMessage() { MessageHexString = ammerterItem.Value.IssuedMessageHexString, MessageId = ammerterItem.Value.IssuedMessageId, FocusAddress = ammerterItem.Value.FocusAddress, TimeDensity = timeDensity.ToString(), }; _ = _producerBus.PublishDelayAsync(TimeSpan.FromMicroseconds(500), ProtocolConst.AmmeterSubscriberWorkerOneMinuteIssuedEventName, tempMsg); //_= _producerBus.Publish(tempMsg); meterTaskInfosList.Add(ammerterItem.Value); } } if (meterTaskInfosList != null && meterTaskInfosList.Count > 0) { //_dbProvider.SwitchSessionPool(true); //await _dbProvider.InsertAsync(meterTaskInfosList); await _meterReadingRecordRepository.InsertManyAsync(meterTaskInfosList, currentTime); } //删除任务数据 await FreeRedisProvider.Instance.DelAsync(oneMinutekeyList); await CacheNextTaskData(timeDensity, MeterTypeEnum.Ammeter); _logger.LogInformation($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集电表数据处理完成"); } /// /// 5分钟采集电表数据 /// /// public virtual async Task AmmeterScheduledMeterFiveMinuteReading() { //获取缓存中的电表信息 int timeDensity = 5; var currentTime = DateTime.Now; var redisKeyList = GetTelemetryPacketCacheKeyPrefix(timeDensity, MeterTypeEnum.Ammeter); var fiveMinutekeyList = await FreeRedisProvider.Instance.KeysAsync(redisKeyList); if (fiveMinutekeyList == null || fiveMinutekeyList.Length <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集电表数据处理时没有获取到缓存信息,-101"); return; } //获取下发任务缓存数据 Dictionary> meterTaskInfos = await GetMeterRedisCacheData(fiveMinutekeyList, timeDensity.ToString(), ((int)MeterTypeEnum.Ammeter).ToString()); if (meterTaskInfos == null || meterTaskInfos.Count <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集电表数据处理时没有获取到缓存信息,-102"); return; } List meterTaskInfosList = new List(); //将取出的缓存任务数据发送到Kafka消息队列中 foreach (var focusItem in meterTaskInfos) { foreach (var ammerterItem in focusItem.Value) { var tempMsg = new ScheduledMeterReadingIssuedEventMessage() { MessageHexString = ammerterItem.Value.IssuedMessageHexString, MessageId = ammerterItem.Value.IssuedMessageId, FocusAddress = ammerterItem.Value.FocusAddress, TimeDensity = timeDensity.ToString(), }; _ = _producerBus.PublishDelayAsync(TimeSpan.FromMicroseconds(500), ProtocolConst.AmmeterSubscriberWorkerFiveMinuteIssuedEventName, tempMsg); //_ = _producerBus.Publish(tempMsg); meterTaskInfosList.Add(ammerterItem.Value); } } if (meterTaskInfosList != null && meterTaskInfosList.Count > 0) { await _meterReadingRecordRepository.InsertManyAsync(meterTaskInfosList, currentTime); } //删除任务数据 await FreeRedisProvider.Instance.DelAsync(fiveMinutekeyList); //缓存下一个时间的任务 await CacheNextTaskData(timeDensity, MeterTypeEnum.Ammeter); _logger.LogInformation($"{nameof(AmmeterScheduledMeterFiveMinuteReading)} {timeDensity}分钟采集电表数据处理完成"); } /// /// 15分钟采集电表数据 /// /// public virtual async Task AmmeterScheduledMeterFifteenMinuteReading() { Stopwatch stopwatch = new Stopwatch(); stopwatch.Start(); //获取缓存中的电表信息 int timeDensity = 15; var currentDateTime = DateTime.Now; var redisKeyList = GetTelemetryPacketCacheKeyPrefix(timeDensity, MeterTypeEnum.Ammeter); var fifteenMinutekeyList = await FreeRedisProvider.Instance.KeysAsync(redisKeyList); if (fifteenMinutekeyList == null || fifteenMinutekeyList.Length <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集电表数据处理时没有获取到缓存信息,-101"); return; } //获取下发任务缓存数据 Dictionary> meterTaskInfos = await GetMeterRedisCacheData(fifteenMinutekeyList, timeDensity.ToString(), MeterTypeEnum.Ammeter.ToString()); if (meterTaskInfos == null || meterTaskInfos.Count <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集电表数据处理时没有获取到缓存信息,-102"); return; } List meterTaskInfosList = new List(); //将取出的缓存任务数据发送到Kafka消息队列中 foreach (var focusItem in meterTaskInfos) { foreach (var ammerterItem in focusItem.Value) { var tempMsg = new ScheduledMeterReadingIssuedEventMessage() { MessageHexString = ammerterItem.Value.IssuedMessageHexString, MessageId = ammerterItem.Value.IssuedMessageId, FocusAddress = ammerterItem.Value.FocusAddress, TimeDensity = timeDensity.ToString(), }; _ = _producerBus.PublishDelayAsync(TimeSpan.FromMicroseconds(500), ProtocolConst.AmmeterSubscriberWorkerFifteenMinuteIssuedEventName, tempMsg); //_ = _producerBus.Publish(tempMsg); meterTaskInfosList.Add(ammerterItem.Value); } } if (meterTaskInfosList != null && meterTaskInfosList.Count > 0) { await _meterReadingRecordRepository.InsertManyAsync(meterTaskInfosList, currentDateTime); } //删除任务数据 //await FreeRedisProvider.Instance.DelAsync(fifteenMinutekeyList); //缓存下一个时间的任务 await CacheNextTaskData(timeDensity, MeterTypeEnum.Ammeter); stopwatch.Stop(); _logger.LogError($"{nameof(AmmeterScheduledMeterFifteenMinuteReading)} {timeDensity}分钟采集电表数据处理完成,共消耗{stopwatch.ElapsedMilliseconds}毫秒。"); } /// /// 电表采集任务指令创建 /// /// 采集频率1分钟、5分钟、15分钟 /// 集中器数据分组 /// private async Task AmmerterScheduledMeterReadingIssued(int timeDensity, Dictionary> focusGroup) { if (timeDensity <= 0) { timeDensity = 1; } if (timeDensity > 15) { timeDensity = 15; } if (focusGroup == null || focusGroup.Count <= 0) { _logger.LogError($"{nameof(AmmerterScheduledMeterReadingIssued)} 电表数据采集指令生成失败,参数异常,-101"); return; } try { //将采集器编号的hash值取模分组 const int TotalShards = 1024; var focusHashGroups = new Dictionary>>(); foreach (var (collectorId, ammetersDictionary) in focusGroup) { if (string.IsNullOrWhiteSpace(collectorId)) { _logger.LogError($"{nameof(AmmerterScheduledMeterReadingIssued)} 集中器信息分组取模失败,无效Key -102"); continue; } // 计算哈希分组ID int hashGroupId = Math.Abs(collectorId.GetHashCode() % TotalShards); // 获取或创建分组(避免重复查找) if (!focusHashGroups.TryGetValue(hashGroupId, out var group)) { group = new Dictionary>(); focusHashGroups[hashGroupId] = group; } // 将当前集中器数据加入分组 group[collectorId] = ammetersDictionary; } if (focusHashGroups == null) { _logger.LogError($"{nameof(AmmerterScheduledMeterReadingIssued)} 集中器信息分组取模失败 -103"); return; } //根据分组创建线程批处理集中器 foreach (var group in focusHashGroups) { await AmmerterCreatePublishTask(timeDensity, group.Value); } } catch (Exception) { throw; } } /// /// 电表创建发布任务 /// /// 采集频率 /// 集中器号hash分组的集中器集合数据 /// private async Task AmmerterCreatePublishTask(int timeDensity , Dictionary> focusGroup) { var handlerPacketBuilder = TelemetryPacketBuilder.AFNHandlersDictionary; var currentTime = DateTime.Now; var pendingCopyReadTime = currentTime.AddMinutes(timeDensity); foreach (var focusInfo in focusGroup) { //构建缓存任务key,依然 表计类型+采集频率+集中器地址,存hash类型 var redisCacheKey = $"{string.Format(RedisConst.CacheTelemetryPacketInfoKey, SystemType, ServerTagName, MeterTypeEnum.Ammeter, timeDensity)}{focusInfo.Key}"; foreach (var ammeterInfo in focusInfo.Value) { var ammeter = ammeterInfo.Value; if (string.IsNullOrWhiteSpace(ammeter.ItemCodes)) { _logger.LogError($"{nameof(AmmerterCreatePublishTask)} 集中器{ammeter.FocusAddress}的电表{ammeter.Name}数据采集指令生成失败,采集项为空,-101"); continue; } //载波的不处理 if (ammeter.MeteringPort == (int)MeterLinkProtocolEnum.Carrierwave) { _logger.LogError($"{nameof(AmmerterCreatePublishTask)} 集中器{ammeter.FocusAddress}的电表{ammeter.Name}数据采集指令生成失败,载波不处理,-102"); continue; } if (ammeter.State.Equals(2)) { _logger.LogWarning($"{nameof(AmmerterCreatePublishTask)} {ammeter.Name} 集中器{ammeter.FocusAddress}的电表{ammeter.Name}状态为禁用,不处理"); continue; } ////排除1天未在线的集中器生成指令 或 排除集中器配置为自动上报的集中器 //if (!IsGennerateCmd(ammeter.LastTime, -1)) //{ // _logger.LogInformation($"{nameof(CreatePublishTask)} 集中器{ammeter.FocusAddress}的电表{ammeter.Name},采集时间:{ammeter.LastTime},已超过1天未在线,不生成指令"); // continue; //} if (string.IsNullOrWhiteSpace(ammeter.AreaCode)) { _logger.LogError($"{nameof(AmmerterCreatePublishTask)} 表ID:{ammeter.ID},集中器通信区号为空"); continue; } if (string.IsNullOrWhiteSpace(ammeter.Address)) { _logger.LogError($"{nameof(AmmerterCreatePublishTask)} 表ID:{ammeter.ID},集中器通信地址为空"); continue; } if (Convert.ToInt32(ammeter.Address) > 65535) { _logger.LogError($"{nameof(AmmerterCreatePublishTask)} 表ID:{ammeter.ID},集中器通信地址无效,确保大于65535"); continue; } if (ammeter.MeteringCode <= 0 || ammeter.MeteringCode > 2033) { _logger.LogError($"{nameof(AmmerterCreatePublishTask)} 表ID:{ammeter.ID},非有效测量点号({ammeter.MeteringCode})"); continue; } List tempCodes = ammeter.ItemCodes.Deserialize>()!; //TODO:自动上报数据只主动采集1类数据。 if (ammeter.AutomaticReport.Equals(1)) { var tempSubCodes = new List(); if (tempCodes.Contains("0C_49")) { tempSubCodes.Add("0C_49"); } if (tempSubCodes.Contains("0C_149")) { tempSubCodes.Add("0C_149"); } if (ammeter.ItemCodes.Contains("10_97")) { tempSubCodes.Add("10_97"); } if (tempSubCodes == null || tempSubCodes.Count <= 0) { _logger.LogInformation($"{nameof(AmmerterCreatePublishTask)} 集中器{ammeter.FocusAddress}的电表{ammeter.Name}自动上报数据主动采集1类数据时数据类型为空"); continue; } else { tempCodes = tempSubCodes; } } Dictionary keyValuePairs = new Dictionary(); foreach (var tempItem in tempCodes) { //排除已发送日冻结和月冻结采集项配置 if (DayFreezeCodes.Contains(tempItem)) { continue; } if (MonthFreezeCodes.Contains(tempItem)) { continue; } var itemCodeArr = tempItem.Split('_'); var aFNStr = itemCodeArr[0]; var aFN = (AFN)aFNStr.HexToDec(); var fn = int.Parse(itemCodeArr[1]); byte[] dataInfos = null; if (ammeter.AutomaticReport.Equals(1) && aFN == AFN.请求实时数据) { //实时数据 dataInfos = Build3761SendData.BuildAmmeterReadRealTimeDataSendCmd(ammeter.FocusAddress, ammeter.MeteringCode, (ATypeOfDataItems)fn); } else { string methonCode = $"AFN{aFNStr}_Fn_Send"; //特殊表暂不处理 if (handlerPacketBuilder != null && handlerPacketBuilder.TryGetValue(methonCode , out var handler)) { dataInfos = handler(new TelemetryPacketRequest() { FocusAddress = ammeter.FocusAddress, Fn = fn, Pn = ammeter.MeteringCode }); } else { _logger.LogWarning($"{nameof(AmmerterCreatePublishTask)} 集中器{ammeter.FocusAddress}的电表{ammeter.Name}采集项{tempItem}无效编码。"); continue; } } //TODO:特殊表 if (dataInfos == null || dataInfos.Length <= 0) { _logger.LogWarning($"{nameof(AmmerterCreatePublishTask)} 集中器{ammeter.FocusAddress}的电表{ammeter.Name}采集项{tempItem}未能正确获取报文。"); continue; } var meterReadingRecords = new MeterReadingRecords() { ProjectID = ammeter.ProjectID, DatabaseBusiID = ammeter.DatabaseBusiID, PendingCopyReadTime = pendingCopyReadTime, CreationTime = currentTime, MeterAddress = ammeter.AmmerterAddress, MeterId = ammeter.ID, MeterType = MeterTypeEnum.Ammeter, FocusAddress = ammeter.FocusAddress, FocusID = ammeter.FocusID, AFN = aFN, Fn = fn, ItemCode = tempItem, ManualOrNot = false, Pn = ammeter.MeteringCode, IssuedMessageId = GuidGenerator.Create().ToString(), IssuedMessageHexString = Convert.ToHexString(dataInfos), }; meterReadingRecords.CreateDataId(GuidGenerator.Create()); keyValuePairs.TryAdd($"{ammeter.ID}_{tempItem}", meterReadingRecords); } await FreeRedisProvider.Instance.HSetAsync(redisCacheKey, keyValuePairs); } } } #endregion #region 水表采集处理 /// /// 获取水表信息 /// /// 采集端Code /// public virtual Task> GetWatermeterInfoList(string gatherCode = "") { throw new NotImplementedException($"{nameof(GetWatermeterInfoList)}请根据不同系统类型进行实现"); } /// /// 初始化水表缓存数据 /// /// 采集端Code /// public virtual async Task InitWatermeterCacheData(string gatherCode = "") { var meterInfos = await GetWatermeterInfoList(gatherCode); if (meterInfos == null || meterInfos.Count <= 0) { throw new NullReferenceException($"{nameof(InitWatermeterCacheData)} 初始化水表缓存数据时,水表数据为空"); } //获取采集项类型数据 var gatherItemInfos = await GetGatherItemByDataTypes(); if (gatherItemInfos == null || gatherItemInfos.Count <= 0) { throw new NullReferenceException($"{nameof(InitAmmeterCacheData)} 初始化水表缓存数据时,采集项类型数据为空"); } //根据采集频率分组,获得采集频率分组 var meterInfoGroupByTimeDensity = meterInfos.GroupBy(d => d.TimeDensity); foreach (var itemTimeDensity in meterInfoGroupByTimeDensity) { //将表计信息根据集中器分组,获得集中器号 var meterInfoGroup = itemTimeDensity.GroupBy(x => x.FocusAddress).ToList(); foreach (var item in meterInfoGroup) { if (string.IsNullOrWhiteSpace(item.Key)) { continue; } var redisCacheKey = $"{string.Format(RedisConst.CacheMeterInfoKey, SystemType, MeterTypeEnum.WaterMeter, itemTimeDensity.Key)}{item.Key}"; Dictionary keyValuePairs = new Dictionary(); foreach (var subItem in item) { keyValuePairs.TryAdd($"{subItem.ID}", subItem); } await FreeRedisProvider.Instance.HSetAsync(redisCacheKey, keyValuePairs); } //在缓存表信息数据的时候,新增下一个时间的自动处理任务,1分钟后执行 TasksToBeIssueModel nextTask = new TasksToBeIssueModel() { TimeDensity = itemTimeDensity.Key, NextTask = DateTime.Now.AddMinutes(1) }; var taskRedisCacheKey = string.Format(RedisConst.CacheTasksToBeIssuedKey, SystemType, MeterTypeEnum.WaterMeter, itemTimeDensity.Key); await FreeRedisProvider.Instance.SetAsync(taskRedisCacheKey, nextTask); } _logger.LogInformation($"{nameof(InitAmmeterCacheData)} 初始化水表缓存数据完成"); } /// /// 1分钟采集水表数据 /// /// public virtual async Task WatermeterScheduledMeterOneMinuteReading() { //获取缓存中的水表信息 int timeDensity = 1; var redisKeyList = GetTelemetryPacketCacheKeyPrefix(timeDensity, MeterTypeEnum.WaterMeter); var oneMinutekeyList = await FreeRedisProvider.Instance.KeysAsync(redisKeyList); if (oneMinutekeyList == null || oneMinutekeyList.Length <= 0) { _logger.LogError($"{nameof(WatermeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集水表数据处理时没有获取到缓存信息,-101"); return; } //获取下发任务缓存数据 Dictionary> meterTaskInfos = await GetMeterRedisCacheData(oneMinutekeyList, timeDensity.ToString(), ((int)MeterTypeEnum.WaterMeter).ToString()); if (meterTaskInfos == null || meterTaskInfos.Count <= 0) { _logger.LogError($"{nameof(WatermeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集水表数据处理时没有获取到缓存信息,-102"); return; } List meterTaskInfosList = new List(); //将取出的缓存任务数据发送到Kafka消息队列中 foreach (var focusItem in meterTaskInfos) { foreach (var ammerterItem in focusItem.Value) { var tempMsg = new ScheduledMeterReadingIssuedEventMessage() { MessageHexString = ammerterItem.Value.IssuedMessageHexString, MessageId = ammerterItem.Value.IssuedMessageId, FocusAddress = ammerterItem.Value.FocusAddress, TimeDensity = timeDensity.ToString(), }; await _producerBus.PublishAsync(ProtocolConst.AmmeterSubscriberWorkerOneMinuteIssuedEventName, tempMsg); //_ = _producerBus.Publish(tempMsg); meterTaskInfosList.Add(ammerterItem.Value); } } if (meterTaskInfosList != null && meterTaskInfosList.Count > 0) { await _meterReadingRecordRepository.InsertManyAsync(meterTaskInfosList); } //删除任务数据 await FreeRedisProvider.Instance.DelAsync(oneMinutekeyList); //缓存下一个时间的任务 await CacheNextTaskData(timeDensity, MeterTypeEnum.WaterMeter); _logger.LogInformation($"{nameof(WatermeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集水表数据处理完成"); } /// /// 5分钟采集水表数据 /// /// public virtual async Task WatermeterScheduledMeterFiveMinuteReading() { //获取缓存中的电表信息 int timeDensity = 5; var redisKeyList = GetTelemetryPacketCacheKeyPrefix(timeDensity, MeterTypeEnum.WaterMeter); var fiveMinutekeyList = await FreeRedisProvider.Instance.KeysAsync(redisKeyList); if (fiveMinutekeyList == null || fiveMinutekeyList.Length <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集水表数据处理时没有获取到缓存信息,-101"); return; } //获取下发任务缓存数据 Dictionary> meterTaskInfos = await GetMeterRedisCacheData(fiveMinutekeyList, timeDensity.ToString(), ((int)MeterTypeEnum.WaterMeter).ToString()); if (meterTaskInfos == null || meterTaskInfos.Count <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集水表数据处理时没有获取到缓存信息,-102"); return; } List meterTaskInfosList = new List(); //将取出的缓存任务数据发送到Kafka消息队列中 foreach (var focusItem in meterTaskInfos) { foreach (var ammerterItem in focusItem.Value) { var tempMsg = new ScheduledMeterReadingIssuedEventMessage() { MessageHexString = ammerterItem.Value.IssuedMessageHexString, MessageId = ammerterItem.Value.IssuedMessageId, FocusAddress = ammerterItem.Value.FocusAddress, TimeDensity = timeDensity.ToString(), }; await _producerBus.PublishAsync(ProtocolConst.AmmeterSubscriberWorkerOneMinuteIssuedEventName, tempMsg); //_ = _producerBus.Publish(tempMsg); meterTaskInfosList.Add(ammerterItem.Value); } } if (meterTaskInfosList != null && meterTaskInfosList.Count > 0) { await _meterReadingRecordRepository.InsertManyAsync(meterTaskInfosList); } //删除任务数据 await FreeRedisProvider.Instance.DelAsync(fiveMinutekeyList); //缓存下一个时间的任务 await CacheNextTaskData(timeDensity, MeterTypeEnum.WaterMeter); _logger.LogInformation($"{nameof(WatermeterScheduledMeterFiveMinuteReading)} {timeDensity}分钟采集水表数据处理完成"); } /// /// 15分钟采集水表数据 /// /// public virtual async Task WatermeterScheduledMeterFifteenMinuteReading() { //获取缓存中的电表信息 int timeDensity = 15; var redisKeyList = GetTelemetryPacketCacheKeyPrefix(timeDensity, MeterTypeEnum.WaterMeter); var fifteenMinutekeyList = await FreeRedisProvider.Instance.KeysAsync(redisKeyList); if (fifteenMinutekeyList == null || fifteenMinutekeyList.Length <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集水表数据处理时没有获取到缓存信息,-101"); return; } //获取下发任务缓存数据 Dictionary> meterTaskInfos = await GetMeterRedisCacheData(fifteenMinutekeyList, timeDensity.ToString(), MeterTypeEnum.WaterMeter.ToString()); if (meterTaskInfos == null || meterTaskInfos.Count <= 0) { _logger.LogError($"{nameof(AmmeterScheduledMeterOneMinuteReading)} {timeDensity}分钟采集水表数据处理时没有获取到缓存信息,-102"); return; } List meterTaskInfosList = new List(); //将取出的缓存任务数据发送到Kafka消息队列中 foreach (var focusItem in meterTaskInfos) { foreach (var ammerterItem in focusItem.Value) { var tempMsg = new ScheduledMeterReadingIssuedEventMessage() { MessageHexString = ammerterItem.Value.IssuedMessageHexString, MessageId = ammerterItem.Value.IssuedMessageId, FocusAddress = ammerterItem.Value.FocusAddress, TimeDensity = timeDensity.ToString(), }; //await _producerBus.PublishAsync(ProtocolConst.AmmeterSubscriberWorkerOneMinuteIssuedEventName, tempMsg); //_ = _producerBus.Publish(tempMsg); meterTaskInfosList.Add(ammerterItem.Value); } } if (meterTaskInfosList != null && meterTaskInfosList.Count > 0) { await _meterReadingRecordRepository.InsertManyAsync(meterTaskInfosList); } //删除任务数据 await FreeRedisProvider.Instance.DelAsync(fifteenMinutekeyList); //缓存下一个时间的任务 await CacheNextTaskData(timeDensity, MeterTypeEnum.WaterMeter); _logger.LogInformation($"{nameof(WatermeterScheduledMeterFiveMinuteReading)} {timeDensity}分钟采集水表数据处理完成"); } #endregion #region 公共处理方法 /// /// 批量获取缓存的表计信息 /// /// 表信息数据对象 /// 采集频率对应的缓存Key集合 /// 采集频率,1分钟、5分钟、15分钟 /// 表计类型 /// private async Task>> GetMeterRedisCacheData(string[] redisKeys, string timeDensity, string meterType) where T : class { //通过lua脚本一次性获取所有缓存内容 var luaScript = @" local results = {} for i, key in ipairs(KEYS) do local data = redis.call('HGETALL', key) results[i] = {key, data} end return results"; var merterResult = await FreeRedisProvider.Instance.EvalAsync(luaScript, redisKeys); //传递 KEYS if (merterResult == null) { _logger.LogError($"{nameof(GetMeterRedisCacheData)} 定时任务采集表数据处理时没有获取到缓存信息,-102"); return null; } // 解析结果(结果为嵌套数组) var meterInfos = new Dictionary>(); ; if (merterResult is object[] arr) { foreach (object[] item in arr) { string key = (string)item[0];//集中器地址对应的Redis缓存Key object[] fieldsAndValues = (object[])item[1];//缓存Key对应的Hash表数据集合 var redisCacheKey = $"{string.Format(RedisConst.CacheMeterInfoKey, SystemType, meterType, timeDensity)}"; string focusAddress = key.Replace(redisCacheKey, "");//集中器地址 var meterHashs = new Dictionary(); for (int i = 0; i < fieldsAndValues.Length; i += 2) { string meterld = (string)fieldsAndValues[i];//表ID string meterStr = (string)fieldsAndValues[i + 1];//表详情数据 T meterInfo = default!; if (!string.IsNullOrWhiteSpace(meterStr)) { meterInfo = meterStr.Deserialize()!; } if (meterInfo != null) { meterHashs[meterld] = meterInfo; } else { _logger.LogInformation($"{nameof(GetMeterRedisCacheData)} 定时任务采集表数据处理时集中器缓存{key}数据的{meterld}处理异常"); } } meterInfos[focusAddress] = meterHashs; } } return meterInfos; } /// /// 指定时间对比当前时间 /// /// /// /// private bool IsGennerateCmd(DateTime lastTime, int subtrahend = 0) { if (DateTime.Now.AddDays(subtrahend) >= lastTime)//当前时间减去一天,大于等于最后在线时间,不再生成该集中器下表生成采集指令 return false; return true; } /// /// 缓存下一个时间的任务 /// /// 采集频率 /// 表类型 /// private async Task CacheNextTaskData(int timeDensity, MeterTypeEnum meterType) { //缓存下一个时间的任务 TasksToBeIssueModel nextTask = new TasksToBeIssueModel() { TimeDensity = timeDensity, NextTask = DateTime.Now.AddMinutes(timeDensity) }; var redisCacheKey = string.Format(RedisConst.CacheTasksToBeIssuedKey, SystemType, ServerTagName, meterType, timeDensity); await FreeRedisProvider.Instance.SetAsync(redisCacheKey, nextTask); } /// /// 获取缓存表计下发指令缓存key前缀 /// /// /// /// private string GetTelemetryPacketCacheKeyPrefix(int timeDensity, MeterTypeEnum meterType) { return $"{string.Format(RedisConst.CacheTelemetryPacketInfoKey, SystemType, ServerTagName, meterType, timeDensity)}*"; } #endregion } }